For centuries, poetry has been used to express the full breadth of the human experience. Now, it’s being used to explore the limits of machine intelligence, too.
This summer, Dartmouth College’s Neukom Institute for Computational Science held its annual Turing Tests in the Creative Arts. Named for the pioneering British scientist Alan Turing, who proposed measuring a computer’s ability to trick a human into thinking it was a person as a gauge of its “intelligence,” the challenge rewarded machine-generated examples of art, literature, poetry and dance that were the most difficult to distinguish from human-created works.
“I find it a really interesting context,” said Dan Rockmore, the Dartmouth mathematics and computer science professor who started the awards. “Everybody always holds out the arts as the thing a computer can’t do. The arts are kind of the goal line for consciousness, sort of an unattainable benchmark.”
A system developed by Thomson Reuters Research Scientist Charese Smiley and Senior Software Engineer Hiroko Bretz took first prize in the poetry contest by creating a sonnet that judges thought most likely to be written by a human.
Smiley said she and Bretz began developing the system in December 2016, using their “innovation time,” or the 10 percent of work hours they can spend on projects that interest them. It took them about six months of working four hours a week or so to complete the system.
Smiley, whose work focuses on computational linguistics, said she was intrigued by the challenge of creating a system that can take a “seed word,” or prompt, and use it to produce creative output.
“How do you make a system that can take any word that’s thrown at it and make it work?” Smiley said. “Not only that, but how can it take what it’s given and make something that not only makes sense, but is appealing? I was very interested by that.”
Bretz said she was intrigued by the idea of whether a computer can create something usually generated by human feeling and expression.
“I liked our winning poem and I also think it is possible for a computer to make a moving and meaningful piece of art,” she said. “But I don’t think a computer can make intentional effort the same way humans do, and it has to depend on coincidence.”
While machine-generated output won the dance, poetry and music categories, there was no winner in the literature category.
“Narrative, I would argue, is a very difficult thing to program,” Rockmore said. “It’s very easy for machines to construct grammatical sentences, to linking words in a theme. Stringing even a paragraph together, though, is really hard.”
That artificial intelligence can create a sonnet (a 14-line poem with 10 syllables per line) but not a short story speaks to the advancement of artificial intelligence today – but not where it may be tomorrow.
“Right now, it’s too hard to give a computer, say, 300 words, ask it to understand them and then have it write the ending,” Smiley said. “But I think we’ll be able to do that sometime in the future.”
Smiley and Bretz aren’t sure what future applications of their system might be, but both said they’d like it if it found a use.
“I really hope we can do something else with it,” Smiley said. “If we could speed it up a little and make some adjustments, it would be fun to put it online and let people play with it.”
The Turing Tests awards for music and dance went to researchers from Indiana University and the Georgia Institute of Technology, respectively.
Below is the sonnet created by Charese Smiley and Hiroko Bretz’s software system:
And be very careful crossing the streets.
How fair an entrance breaks the way to love!
Left, doors leading into the apartments.
Just then a light flashed from the cliff above.
The fields near the house were invisible.
Objects of alarm were near and around.
The window had only stuck a little.
From the big apple tree down near the pond.
The large cabin was in total darkness.
Come marching up the eastern hill afar.
When is the clock on the stairs dangerous?
Everything seemed so near and yet so far.
Behind the wall silence alone replied.
Was, then, even the staircase occupied?
Our Cognitive Computing Center of Excellence focuses on exploring the rapidly developing field of cognitive computing and machine intelligence. Learn more about how cognitive computing is transforming knowledge work.