Sociotechnical Map

This project looks at predictive emoji as a Sociotechnical System for Computing Emotion.

 

Predictive emoji is a feature which has been integrated into the many of the mobile keyboards on smartphones and tablets. This feature automatically pulls up related emojis based upon the words a user types out in plain text. The incorporation of emoji prediction into many smart keyboards has greatly simplified the process of adding emojis into electronic communications. What was once a seconds or minutes long search for appropriate emojis has now been condensed into mere milliseconds, as the predictive emoji algorithm parses the users’ words and draws upon its machine learning algorithm to present the user with the best emoji candidates. 

            

The first emojis came into being during the 1990s, as new models of cellular telephones with graphic interfaces made it possible to see text on the screen. The first color emoji set was designed in 1999 by graphic designer, Shigetaka Kurita, for Japan’s main mobile carrier, Docomo. Most of these emoji were designed to be purely informational (weather icons, zodiac icons, and the like), but a select few, such as a set of heart emoji, could be used to convey the more complex emotional aspects of speech. These heart emoji were expressly designed by Kurita to lend some warmth to cold and sterile new forms of communication, such as email and SMS. 

​

 

 

 

 

 

 

 

 

 

Shigetaka Kurita's Original Emoji Designs (Published by MoMa)

Emoji quickly took off in Japan and began to be integrated into competing platforms. New emoji were added, many of which depicted simplified facial expressions. Soon, internet users across the globe began to adopt emoji into their lexicon, and large tech companies were beginning to take notice. The Unicode Consortium decided to index 625 emoji characters in 2010 after they were petitioned by engineers at Google and Apple. Apple would develop their emoji keyboard in 2011, followed by Android in 2013, making emojis easily accessible to their user-base, which led to an explosion in their popularity. 

 

The first application touting predictive emoji, "Emoji Type," was launched in 2014, followed by Apple iOS and Microsoft Swiftkey in 2016. This latest stage of integration illustrates how vital emojis have become in modern digital communication. A recent study revealed that 92% of internet users have emoji in at least some of their communications. At the time of this writing, 3,633 emojis have been assimilated into the Unicode Standard, and this list is bound to expand as organizations and individuals continuously submit new emojis to the Consortium to be encoded. 

 

Emojis have long served to fill a void in computer mediated communication (CMC), standing in for the facial expressions and body language otherwise lost in text-based communications. Shigetaka Kurita admits that he included heart emojis in his original emoji set for this reason precisely, “If someone says Wakarimashita [in text] you don’t know whether it’s a kind of warm, soft ‘I understand’ or a ‘yeah, I get it’ kind of cool, negative feeling…you don’t know what’s in the writer’s head.” Predictive Emoji takes this original intent one step further by automatically suggesting emojis which could provide a sort of context clue. These emojis can be used to clarify tone, express humor or sarcasm, soften or harden statements, and perform other sorts of emotional work which would otherwise go unexpressed in plain text messages. As more of our relationships take place by means of CMC, more of this emotional work is being delegated to these icons, with decidedly mixed results. 

 

Though emoji has been heralded as an “universal language,” bonding users across the globe. Emoji usage and interpretation is deeply bound to the users’ culture, language, identity, and age. The smiling emoji, 🙂, is usually used to denote happiness, but has been appropriated by young Chinese users as a symbol of pain and alienation. The thumbs up emoji, 👍, a benign symbol of assent in most of the world, can be considered obscene in Greece and much of the Middle East. The ever popular, “crying while laughing emoji,” 😂 was named the word of the year by Oxford English Dictionary in 2015 and is still the most commonly used emoji to denote laughter, but is frequently derided by teenagers and young adults as an “old person” emoji. The list could go on and on. These examples don’t even take into account how various media platforms (eg. Twitter, Apple, Google, ect.) render emoji glyphs in different ways, potentially obscuring the user’s intended meaning even further. This state of affairs reflects the ambiguity inherent within these symbols, as emoji, much like their old-fashioned analog counterparts--facial expression and body-language--can be misconstrued in all sorts of ways. Reflecting upon his famous creations, Shigetaka Kurita relented that, "I think the heart symbol is probably used the same way by everyone, but then there are probably things that only Japanese people would understand, or only Americans would understand… it would be great if we could compare, and have that lead to people starting to use things in the same way." Thanks to developments in predictive emoji that dream may be possible in the near future.

 

Predictive emoji seeks to remedy this problem by turning to the field of natural language processing (NLP), a discipline within the field of machine learning, to better predict how a given user will deploy emojis. These NLP algorithms use sentiment analysis to plumb user data for indicators which could be used to predict emotional state and conversational style (be sure to check out our Technical Component Page for more information about how this is done). Buoyed by recent advances in artificial intelligence, the capabilities of emoji prediction algorithms are greatly expanding, as researchers are currently developing ways to parse spoken language, as well as facial expressions, to produce fitting emojis for the user. These developments have profound implications for the future as AI (and the companies which helped to create that AI) become better at understanding and predicting our emotions—leaving us to wonder, what will happen now that emotion has become computable?

​

Authored by Megan Hearst (April 2022)