July/August 2018

More articles in this issue:

• Pay Dirt
• LiDAR Lights the Way
• Cryptocurrency Is Impacting Traditional Sports/Esports
• iRobot’s Colin Angle
• There’s No Place like a Smart Home

Artificial intelligence (AI) software is editing films and TV programs, writing music and handling cinematography. Studios, networks and production companies are also exploring ways to use AI in script writing and distribution workflow. Creative ways to use AI, machine learning and companion technologies are just beginning, often concerning humans who thought their creative talents could not be replaced by machines. Yet scripted programming and music on streaming platforms are suited to computer-assisted creativity.

So far, AI's output has been on routine tasks. For example, Sony's Computer Science Laboratories (CSL) in Paris has created "Flow Machines" to help musical artists with their creative process, although the software has also written songs on its own. At Google, the "Magenta" research project developed "NSynth Super" to see how machine learning tools can help artists create art and music in new ways.

IBM's Watson AI system generated highlight clips for the CBS coverage of the Masters Golf Tournament and was also used to package a preview trailer for the 20th Century Fox movie Morgan in 2016. In both cases, Watson identified the exciting segments of the full production and stitched them together into a template.

Since AI is based on big data, the systems look for scenes that audiences like, then match them with actual production: a computer-age blend of marketing and media. 

Observers say the first steps into  AI creative productions will involve collaborations between experienced staffs and the technology that automates the process. Meanwhile, labor unions remain cautious of the tech intrusion and threat to their jobs. 

Beyond the first wave of AI activity looms a larger prospect of creative computerization. For example, cinematographic drones today may be "piloted" by human camera operators. But as the drones learn how to follow a certain character or sweep above an action scene, the need for a cinematographer may become redundant.

Then There's Writing

AI software already "writes" reports for the Associated Press and other media groups on financial news and sports. It digests the data (stock earnings or baseball stats, for example) and puts it into a familiar style for that coverage. 

Creative writers acknowledge there are only a handful of fictional storyline structures, and AI systems have learned these fundamentals. The Massachusetts Institute of Technology Media Lab and McKinsey’s Consumer Tech and Media team collaborated on a project in which neural networks watched thousands of movies, TV shows and online videos. The machine-learning process created a vast database of plot situations and identified "emotional arcs" – relationships between characters, plot dynamics, even the music and visual imagery that appeal to audiences, based on what professional storytellers use. 

Armed with such knowhow, computers are ready to create entire scripts leading groups like the Writers Guild of America to closely monitor developments. Most experts agree AI use will be collaborative, involving human and machine-made creative participation. At least for now. 

That begs the question, "Where does creativity come from?" Sony's CSL has explored "styles" in musical performance, composing and painting. It found that artists are "influenced" by others as they develop their own unique approach. Some experts say AI systems will follow similar creative "style" paths. 

David Leibowitz, former Recording Industry Association of America vice president and general counsel, says, "Music in film and TV conveys specific feelings to correspond to the particular scene. AI can analyze vast libraries of film and production music to produce AI-generated music based on mood, emotion, genre, reference decade, tempo and lyrical summary."

That leaves room for machine-made creativity, albeit facing as Leibowitz says, "thorny rights, ownership and licensing issues." And we still haven't truly figured out the impact that AI will have on directors, producers and performers.