Like most art forms, motion picture editing is far more difficult than it appears on the surface. A good editor uses a sequence of moving images and sound to generate meaning and elicit emotion from the audience. The process is a far cry from just slapping images together.
Sergei Eisenstein, a Soviet film editor in the early half of the last century, was fascinated about how editing could alter the perception of a story for the audience. He eventually created the Soviet montage theory which has been featured in articles and books — Film Form and The Film Sense being the best.
Eisenstein came along in a unique period of history when innovation in aesthetics, storytelling and technology were merging in the cinema. It was a time period of great experimentation in film.
“How grateful I was to fate for having subjected me to the ordeal of learning an oriental language, opening before me that strange way of thinking and teaching me word pictography. It was precisely this “unusual” way of thinking that later helped me to master the nature of montage, and still later, […] helped me to comprehend the most recondite (obscure) in the methods of art,” Eisenstein wrote in his book, Notes of a Film Director (1946).
Eisenstein’s montage theory boils down to this: a series of connected images allows complex ideas to be conveyed to an audience. When edited together, a montage of these images constitutes the entirety of a film’s ideological and intellectual power.
Eisenstein cited five methods of montage in his films: metric, rhythmic, tonal, overtonal and intellectual. He describes in detail each of these editing methods in his work, “Film Form.” They are too complex to get into in this article.
Though Eisenstein wasn’t the first nor the last of the editing theorists, his techniques are still taught in film schools all over the world. From Charlie Chaplin to Alfred Hitchcock, film directors have used Eisenstein’s theories in editing their most popular films.
When studying Eisenstein or others, it should be remembered that the theoretical approaches to editing are just as important as the practical realities of working in a modern edit bay. It is in these theories that the editor unearths the genuine art form of how to use images and sound to tell a story.
Though editors are often considered the architect of their films, editing jobs are now threatened by the increase in the power of computers. In the early days of cinema, some editors created complex mathematical formulas for editing. Today, powerful hardware and software, along with programming languages, have generated far more sophisticated storytelling techniques.
A popular technique that’s been used for many years is algorithmic editing, which means mathematically editing video to a specific plan. An example of algorithmic editing is the basic flicker film — where black and white shots are alternated rapidly one after the other. This repetition creates a very uncomfortable effect on the audience. Of course, this example is oversimplified since many algorithms developed in software contain random variables that change the equation.
Modern algorithmic editing is computer driven. Editors can achieve different outputs every time since algorithms now contain random or pseudo-random processes. In addition, the computer can be programmed to produce results that are unexpected — a process that is often exploited by artists creating algorithmic art.
When editing on the latest editing software, video footage is seen as timeline-based clips. In algorithmic editing, the editor is not accessing the raw footage in the same way. Here the editor has direct access to any frame or clip within a location.
It becomes possible to use the variables existing in a section of footage, such as pixel variation and audio deviations, to determine where to make cuts and how to put the video back together again.
The model for such edit points can be a music score. The editor can use an original music score and write an algorithm that parses a database of video looking for images that match the notes.
Today, algorithms are being used to develop new software to automate editing, such as with action camera footage. Disney Research has devised a tool that can automatically edit the final footage together with an intuitive, human-style editing approach. It has created software that splices together footage shot on different devices with remarkable results (no human help needed).
Most of us prefer to stick to the human approach for editing, but algorithms, combined with artificial intelligence (AI), are already ubiquitous in the edit suite and getting more powerful. Mathematical calculations already determine upscaling, render optimization and keyframe interpolation. AI and machine learning such as Adobe’s Sensei is taking editing to a new automated level.
Though editing still depends on human creativity, that may not last long if technology continues at the current rate of development. Computer-assisted editing techniques will soon use software to automatically edit different versions of a story and sensors attached to test audiences can register the reaction to every frame. The day may soon come when perfect montages can be made without human interaction.
It’s a scary scenario that all of us must consider.