“Computational Video Editing for Dialogue-Driven Scenes” may make human video editors obsolete.

All images via Stanford University and prosthetic knowledge.

Remember the good old days when film editing involved physically pulling film stock by hand? Computers and digital filmmaking breakthroughs have, up to this point, been welcome advances that help an editor craft his or her story. Well, that all may be about to change.

Not to scare all the professional video editors out there, but with understandable hesitation, let’s look into some groundbreaking new science displayed in a recent research project from Stanford University and Adobe Research.

Meet the Computational Video Editing Technology

We present a system for efficiently editing video of dialogue-driven scenes. The input to our system is a standard film script and multiple video takes, each capturing a different camera framing or performance of the complete scene. Our system then automatically selects the most appropriate clip from one of the input takes, for each line of dialogue, based on a user-specified set of film-editing idioms

In the video (and in the quote above), we get a glimpse into the comprehensive approach these researchers took to the technology. The tech can follow a script, sort through multiple takes, and even apply lessons learned from film editing idiom textbooks to create nuanced and well-edited, dialogue-driven scenes.

Automatic Labeling

Future Tech: Computers That Edit Dialogue-Driven Scenes — Automatic Labeling

Think the technology is creepy enough? Well, the “automatic labeling” algorithm allows the technology to categorize and instantly recall every take (including audio, video and text analysis) to comprehensively label each clip for later use. You can see an example of the technological breakdown below.

Future Tech: Computers That Edit Dialogue-Driven Scenes — Technological Breakdown

Idiom Building Editing

Future Tech: Computers That Edit Dialogue-Driven Scenes — Idiom Building

The video also showcases the technology’s “Idiom Builder” panel, which, after it labels, organizes, and prioritizes each clip, allows you (or the program) to apply film school idioms to dictate the feel of the film’s narrative. Concepts such as “start wide,” “peaks and valleys,” and “jump cuts” are not lost on the technology.

While one could argue that this technology is meant to help video editors save time by streamlining their workflow, it does open up debate about where we should draw the line between art and technology — which can be scary, especially if we find that there doesn’t seem to be one at all.


What are your thoughts on “Computational Video Editing” technology? Let us know in the comments.