Predictive text AI: a “co-writer” that can save users 2 billion keystrokes a week
by Pisana Ferrari – cApStAn Ambassador to the Global Village
In a very entertaining article for a recent issue of The New Yorker author and journalist John Seabrook recounts his experience with predictive text AI, and, in particular, with “Smart Compose”, a feature introduced by Google in 2018. (1) Smart Compose suggests endings to one’s sentences, as one types them, by harnessing the predictive power of deep learning, a subset of machine learning. (2) Based on the words one has written, and on the words that millions of Gmail users have followed those words with, Smart Compose guesses where your thoughts are likely to go next. Hit “tab”, Seabrook says, and you’ve saved yourself as many as twenty keystrokes. About one quarter of an office worker’s day is taken up by e-mail, according to a study by McKinsey quoted in the article: Smart Compose saves 2 billion keystrokes a week.
Seabrook says he was fascinated by how frequently the AI was able to accurately predict his intentions and even have a better idea than he did. In one particular instance while he was typing an e-mail to his son which began with the words “I am p-“, and he was about to write “pleased”, predictive text suggested “proud of you.” “Clearly Smart Compose thinks that’s what most fathers in my state say to their sons in e-mails”, he writes. But, sitting at his keboard, he says he could feel his skin prickling on the back of his neck. “It wasn’t that Smart Compose had guessed correctly where my thoughts were headed-in fact, it hadn’t. The creepy thing was that the machine was more thoughtful than I was”.
Seabrook also writes about his later experience with GPT-2, a “supercharged” version of Smart Compose, developed in 2019 by OpenAI, trained on a 8 million web pages. (3) With only a couple of sentences to establish a theme and a tone for the article, the AI could “write” entire paragraphs of text. The company has not released GPT-2 yet, over concerns about potential malicious applications (4), but Seabrook was able to experiment with a version trained on The New Yorker archive of articles. (5) Some things sounded right, and the AI had the ability to take on some of color of the prompt’s prose, he says, but it could not tell between fiction and non fiction, and there were gaps in “the kind of common sense knowledge that tells you overcoats aren’t shaped like the body of a ship”, he says. The resulting prose sounded “as though the writer had fallen asleep and was dreaming”. Seabrook’s initial excitement gave way to queasiness and he writes that it hurt to see the rules of grammar and usage, which he lived his entire writing life by, “mastered by an idiot savant that used math for words”.
What if a later version GPT-2, more powerful than the current one, asks Seabrook, were able to write causally and distinguish truth from fiction and at the same time draw from its well of deep learning? He says one could imagine a kind of “Joycean superauthor, capable of any style, turning out spine-tingling suspense novels or massively researched biographies”. Humans would stop writing, or at least publishing, because all the readers would be captivated by the machines. What then? GPT-2, prompted by Seabrook on the above paragraph predicted – fittingly, we say – the next sentence: “In a way, the humans would be making progress.”
Footnotes
- “The Next Word. Where will predictive text take us?”, John Seabrook, The New Yorker, October 14, 2019: https://bit.ly/2NHkTPj
- Smart Compose: https://ai.google/research/pubs/pub48231/
- OpenAI is a non profit AI company founded in 2015 in San Francisco, California. Its mission is to ensure that artificial general intelligence benefits all of humanity: https://openai.com/
- “Better Language Models and Their Implications”, Open AI: https://bit.ly/36AxwV5
- GPT-2 was trained on the The New Yorker archives but this data was used only for the purposes of the experiment reported in the article.