AI system reads novels, writes music for them
- 15 May, 2014 21:42
What would "Alice in Wonderland" sound like if it were set to music? And what if the composer were a computer?
It's not something you might think of putting on your MP3 player, but researchers have trained an artificial intelligence system to read works of fiction and create music based on the texts. TransProse is a project that uses computers to translate literature into music.
First, the system reads the text of a work such as "Peter Pan." Based on word scans, it assigns "densities" of two different states, positive or negative, and eight different emotions -- joy, sadness, anger, disgust, anticipation, surprise, trust and fear.
Then, it sets about composing a musical piece that chronologically follows the novel, broken up into beginning, early middle, late middle, and end parts.
The "emotion density data," as the researchers describe it, is used to determine the tempo, key, notes, octaves and other musical variables. Shorter notes, for instance, correspond with more emotionally dense areas of the narrative.
So what does literature-inspired AI music sound like?
"The Adventures of Sherlock Holmes," the detective classic by Arthur Conan Doyle, was assigned "trust" as its highest emotion, followed by "fear."
It's a simple, pretty but fairly bland piano composition in C major with a few trills that might evoke Mozart. But listeners expecting a hint of Holmes' criminal adversaries will be disappointed.
Joseph Conrad's "Heart of Darkness" is rendered as a brooding progression in C minor with "fear" and "sadness" as its themes. It may capture the atmosphere of the narrator's journey into colonial Africa, but it's not exactly a moving work.
"The current version of TransProse is just the beginning of our investigation and we don't claim to be making beautiful music at this point," Hannah Davis of New York University wrote in an email.
"This iteration is a starting point to see if we could programmatically translate the basic emotions of a novel into a musical piece that holds the same basic emotional feeling, which I think has been pretty successful."
TransProse grew out of a thesis project by Davis and a collaboration with Saif Mohammad, a researcher at the National Research Council Canada who had created a word-emotion lexicon.
The pair presented a paper on their work at a conference in Sweden last month, and plan to explore the technology by representing characters in novels as motifs in music.
"There are many creative ways in which going from text to music and from music to text can be used," Mohammad wrote in an email.
"One practical application might be in online book stores where a customer can click on a button to listen to the emotional tone of a book before deciding to buy it."