Research

• Cohn, N., Engelen, J., & Joost, S. (2019). The Grammar of Emoji? Constraints on Communicative Pictorial Sequencing: Principles and Implications. Cognitive Research, 4(1) doi:http://dx.doi.org/10.1186/s41235-019-0177-0

 

This study examines emoji as an “emerging language,” by looking at how people use emojis in two experiments; one where they only communicate using emojis, another where they must substitute an emoji for at least one word in their sentences. This study highlights how expression through emoji is limited as it depends upon a set number of “codified whole-unit signs,” bereft of the versatility and nuance present in other language forms. This study illustrates the limitations of this technology as it moves forward (Authored by Megan Hearst).

​​​

• Gibson, W., Huang, P., & Yu, Q. (2018). Emoji and communicative action: The semiotics, sequence and gestural actions of ‘face covering hand.’ Discourse, Context & Media, 26, 91–99. https://doi.org/https://doi.org/10.1016/j.dcm.2018.05.005 

 

This article use conversation analysis to investigate the role of emoji as a communicative phenomenon in the development of text-talk by examining their relationship to other textual acts in a Chinese mobile reading community. It also analysis the importance of emojis in texting, and the relationship between emoji gestures and face-to-face interaction. (Authored by Xiyue Hu）

​

• Juang, B.H., and Lawrence R. Rabiner. Automatic Speech Recognition - A Brief History of the Technology Development. 2004, https://folk.idi.ntnu.no/gamback/teaching/TDT4275/literature/juang_rabiner04.pdf 

​

This article provides a historical background for the development of automatic speech recognition during the last few decades from the 1930s system model, 1980s statistical model, to the widespread use of speech recognition systems today including introductions for each system or model algorithm, fundamental development processes, and challenges for the ongoing researches and designs. （Authored by Xiyue Hu)

​

• Khalil, R. A., Jones, E., Babar, M. I., Jan, T., Zafar, M. H., & Alhussain, T. (2019). Speech emotion recognition using Deep Learning Techniques: A Review. IEEE Access, 7, 117327–117345. https://doi.org/10.1109/access.2019.2936124ds

 

The approach for speech emotion recognition (SER) primarily comprises three phases known as signal preprocessing, feature extraction, and classification. In the first phase,  the Acoustic preprocessing would do the work of denoising, as well as segmentation so that the machine could determine the meaningful units of the signal. The feature extraction is utilized to identify the relevant features available in the signal. Lastly, the mapping of extracted feature vectors to relevant emotions is carried out by classifiers. (Authored by Ruhan Hou)

​

• Kralj Novak P, Smailović J, Sluban B, Mozetič I (2015) Sentiment of Emojis. PLoS ONE 10(12): e0144296. https://doi.org/10.1371/journal.pone.0144296

​

This study analyzes sentiment across a wide array of tweets, researching how emojis are used, and what sentiments they are used for. By utilizing human annotators to label 1.6 million tweets, the project sought to establish an emoji lexicon, namely Emoji Sentiment Ranking, where each emoji is rated with a specific sentiment, from negative to neutral and to positive. It effectively established the very first emoji lexicon, so that researchers and scholars (and likely even tech companies) can use it to understand how emojis are really being used in real-time on Twitter. This project links directly with our attempt at deblackboxing predictive emojis and analyzing how tech companies attribute emotions to the images. (Authored by Andrew Peacock)​

​

• Medhat, W., Hassan, A., & Korashy, H. (2014). Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal, 5(4), 1093–1113. https://doi.org/10.1016/j.asej.2014.04.011

​

There are many models for sentiment analysis. However, they all go through a similar process. First of all, they choose a database to use. Then they all identify whether the sentence is subjective or objective. Then they just break down the sentence to detect the emotion and then classify the sentiment into the existing model.  (Authored by Ruhan Hou)

​

• Morstatter, Fred; Shu, Kai; Wang, Suhang; Liu, Huan. 2016. Cross-Platform Emoji Interpretation: Analysis, a Solution, and Applications. In Proceedings of ACM Conference, Washington, DC, USA, July 2017 (Conference’17), 9 pages. DOI: 10.1145/nnnnnnn.nnnnnnn.

​

​This study introduces a vastly relevant topic to our repertoire in the form of how emojis are interpreted for different emotions across platforms which render their emojis in opposing ways. The project looks at iOS, Windows, Android, and Twitter for differences in sentiment via sentiment analysis and emoji mapping according to tweets and their emojis. They find that the difference in emojis semantics is statistically significant across platforms, that people use them differently across platforms, and that researchers would do best by canceling out the “emoji noise” (9) entirely, removing the emojis rather than dealing with their copious polysemic differences. That project will inform us greatly in terms of providing us contextual information on how emojis may be attributed meaning differently across emojis AND platforms. We shall weave this important evidence into our understanding of emoji sentiment classification greatly. (Authored by Andrew Peacock)

​

• Neelima, G., Subha, K., Chigurukota, D., Kumar, B. (2021). Automatic Sentiment Analyser Based on Speech Recognition. Turkish Journal of Computer and Mathematics Education. Vol.12 No.10 (2021), 5388-5398.    

 

​There are three types of emotion detection of speech. The first is based on text. The second is via sound. And the third one is based on facial expression. To analyze the emotion, we should firstly collect data (people speak). Then use the MFCC and Chroma techniques to extract the features of the audio and later on, we use MLP classifier techniques to classify the emotion and give it to the recommender system which recommends the user with the appropriate emojis. (Authored by Ruhan Hou)

​

• Pardes, A. (2018). The WIRED Guide to Emoji. Wired. Retrieved 2022, from https://www.wired.com/story/guide-emoji/ 

 

“The Wired Guide to Emoji” is an excellent jumping off point for individuals who want to learn more about this subject. This article recounts the history of emojis, gives an overview of Unicode’s role in emoji development, and explains how emojis became an ingrained part of modern digital culture. NOTE: This article was written in 2018, so statements made about the current state of emoji are based on Unicode 10 (Authored by Megan Hearst).

​

• Paggio, P., & Tse, A. (2022). Are Emoji Processed Like Words? An Eye-Tracking Study. Cognitive science, 46(2), e13099. https://doi.org/10.1111/cogs.13099

 

In this study, researchers utilized eye tracking technology to examine how people read emojis as an integrated part of the text. Emojis were either substituted for nouns in a sentence, or added as an addendum to statements, and study subjects’ reading speed and comprehension were analyzed. This analysis revealed that emojis significantly slowed their reading speed and were difficult to integrate with plain text characters. This result shows that while emoji replacement is a popular and effective way to catch the reader's eye, this lexical choice actually decreases reader comprehension. (Authored by Megan Hearst).

​

• Singh, G. V.; Firdaus, M.; Ekbal, A.; Bhattacharyya, P.  (2022). Unity in Diversity: Multilabel Emoji Identification in Tweets. IEEE Transcations on Computational Social Systems. https://doi-org.proxy.library.georgetown.edu/10.1109/TCSS.2022.3162865

​

​Singh et al (2022) present an incredibly relevant and informative study, based upon polysemic emoji prediction datasets. They attempt to attribute and predict the best emojis for a set of 600,000 Tweets while referencing recent work and presenting the current problems in the field. They reference issues in attributing multiple meanings to emojis, such as problems with sarcasm and giving meaning based upon single words, even if the single words’ meanings do not relate with the sentences’ or documents’ larger meaning. However, the article is incredibly informative within the field of NLP and emoji prediction models and will provide our research with an essential reference as we unpack the emoji prediction technologies that exist within our current text-messaging and social media software. (Authored by Andrew Peacock)

​

• Zhang, Y., Wang, M., & Li, Y. (2021). More than playfulness: Emojis in the comments of a WeChat official account. Internet Pragmatics, 4(2), 247-271.

 

This article focuses on examining the use of Wechat, a Chinese major communication tool, which has unique designs of emojis. It analyzes how the use of emojis helps express an individual’s emotions and felling to fulfill online communication. Another major part of this article is quantitative analyses of emoji in use, which include elf-disclosure, self-praise, humor, and complaining. (Authored by Xiyue Hu)

​

• Zeng, Jing; Chan, Chung-hong; Schäfer, Mike S. (2022) Contested Chinese Dreams of AI? Public discourse about Artificial intelligence on WeChat and People’s Daily Online, Information, Communication & Society, 25:3, 319-340, DOI: 10.1080/1369118X.2020.1776372

​​

​This research project analyzes how Chinese citizens utilize social media so that it may serve as a counter-public against the Chinese Dream or its goal of establishing itself as an AI authority worldwide. The study uses sentiment analysis to measure whether citizens are using the platforms of WeChat and People’s Daily Online to spread controversial sentiments about the political aims of the Communist Party in China. From 2015 to 2018, the researchers pulled 128,343 WeChat articles and 20,666 PD articles, seeking differences in sentiment as compared to the Chinese authorities. The researchers found that civil society groups instead served as an amplifier of political and economic frames. Since Chinese civil society groups, such as WeChat, rely on the government for existence, they need a close and harmonious relationship with authorities and do not challenge them. This project greatly informs the context surrounding how software such as WeChat can be severely limited and censored due to the power of national authorities. (Authored by Andrew Peacock)

​​