Generative Film and Theories of Montage

Introduction

Montage theory has long had an important effect on both commercial and experimental filmmakers and moving image artists. In recognizing the importance of the theory and practice of montage within the cinematic tradition, and while acknowledging these best known examples of montage theory, a question for generative artists might be: What kinds of montage or what kinds corollaries to traditional theories of montage might be observed in algorithmic film?

This paper discusses the intentions and meanings of traditional cinematic montage in an effort to learn whether those principles and techniques have parallels, extensions, or corollaries that might be accessible through a system of montage based on computation and algorithm, but especially through the effects of sequencing single rames of video through the use of a generative process such as a context free grammar.

The visual material in this description comes from a group of generative films I have made titles: Three Hollywood Grammars: Chase, The Shootout, and The Conversation. The generative films use classic Hollywood productions as examples of specific cinematic sequences or 'Hollywood grammars'. These grammars were: the A/B shot of the walk and talk conversation in “The Conversation”, the parallel intercutting of gritty realism's good guy-bad guy shootout in “The Shootout”, and the classic urban chase scene as choreographed for city streets in “Chase”. The classic films were deconstructed as databases of single frames grouped by shot. Then an algorithmic or generative 'movie engine' that uses context free grammars to sequence single frames of video into long instances of cinematic montage re-assembled these deconstructed frames into 'new' or 're-imagined' montage. These films have been shown at the Millennium Film Workshop in New York and at the 2015 International Conference of Generative Art in Venice, Italy.

Traditions of Cinematic Montage

For many filmmakers, montage, or the process of juxtaposing one frame of film against another is the essence of moving image art. Russian psychologist Lev Kuleshov was one of the first to study cinematic juxtaposition and propose a theory of montage. Later, Russian filmmaker Sergei Eisenstein would develop an extensive theory on the ways a series of images can arranged or cut together to produce abstract concepts in the minds of viewers. Dziga Vertov also believed in the ideological potentials of montage. Hitchcock felt it was film montage that created both meaning and emotion in an audience. A digital media theorist, Lev Manovich cites experiments in film montage as an important basis for new media aesthetics.

Yet each of these artists and theorists have different ways of organizing and executing montage and therefore hold differing ideas both on how montage operates and on what makes montage effective. Kuleshov focuses his efforts on he analysis of the emotional and psychological effects of shot sequences, noticing that the order of shots are what produce meaning in film. A shot of a gun going off followed by a shot of a woman screaming, Kuleshov notices, is a completely different story than a shot of a woman screaming followed by a shot of a gun going off. Hitchcock describes montage as pure film, as the only way of creating the emotional impact of a story on screen. With its legendary seventy camera sets-ups and ninety edits, it is probably the violence of the montage that persuades the audience of the horror of Psycho's famous shower scene, not the actual frame content of the shots themselves. Vertov uses montage to create oddities, force unlikely comparisons, and make visual jokes. Vorkepich uses similar formal relationships in the montage of visual objects to create perceptual experiences such as an object or character's qualities changing over a series of events or a period of time. German directors like F.W. Murnau and Robert Wiene used montage to externalize the interior feelings and motivations of their characters, cutting odd reflections, mirror images, or hyper-focalizations from everyday life into their scenes to build a system of association that attempts the expression of the human subconscious. Eisenstein, a student of the abstraction of Japanese ideograms, posits five types of juxtaposition in montage: metric montage which uses the pace of film edits to create meaning, rhythmic montage which uses editing to express progressions in some aspect of character or story, tonal montage which elaborates on the inner feelings of the character under consideration, overtonal montage which elaborates on the emotional associations a character might provide in a scene, and intellectual montage which uses juxtaposition as a linking device, for example, as a way to intercut a soldier's face with a statue of a lion, associating the soldier with strength and nobility, and giving audiences a working metaphor for the meaning of what they are watching onscreen.

Role of the Generative

Often composed frame by frame. algorithmic editing, in this case, generative montage is based on computer code's ability to access any single frame regardless of its position in a sequence. Rather than working in support of a narrative of filmic plot, algorithmic cinema assembles sequence according to procedural rules which can change as their programs is run, allowing frames to be ordered dynamically according to generative ideas. Much of the aesthetics of algorithmic film is inherited from non-objective and structuralist experiments. But certain attributes of montage can seem even more critical to generative filmmakers who use the speed and complexity of computation to analyze, sequence, patten, alter, iterate, and construct moving image sequences.

In the movies discussed here, montage was assembled through the application of a context free grammar or CFG. CFGs are a set of generative production rules that, in this case, create long complex patterns of film frames according to specific symbols. A start symbol is a special non-terminal system that initiates the pattern sequence. This start symbol is then followed by a non-terminal symbol. The non-terminal symbol is then replaced by following a set of productions rules which determine what pattern of terminal symbols can be rendered to the screen. A cinematic generative montage is created by beginning with a start symbol, then interpreting a non-terminal symbol by applying the production rules of a particular CFG in the production of a set of terminal symbols and, finally, rendering the
resulting sequence of frames denoted by those terminal symbols to the screen as generative algorithmic montage.

Constructing such a dynamic or generative sequence is not possible with celluloid and editing block due to the fixed nature of celluloid and even traditional digital film techniques. However, while computers are in most instances associated with algorithmic and procedural montage and generative processes, machines are not, in and of themselves, essential to generative art and computers are not fundamentally necessary to the processes of algorithmic filmmaking. A large number of experiments in moving image, done by hand, with an editing block could, in fact, be considered algorithmic in strategy. While adding power and complexity, computer software, it is important to note, is in many ways no more than an extension of an earlier artistic vision and cinematic tradition.

However, in a live or dynamic assembly of shots, computers do add variation and surprise to montage unavailable to traditional artists. Generative montage often contains random, noisy, or evolutionary processes. Viewers and creators of generative montage should not ever expect to see the same film twice. Algorithmic films can be written to produce unexpected juxtapositions and sequence compositions that lie outside the rational, and both characteristics are frequently among the creative goals of algorithmic filmmakers. Generative montage can also be designed to allow or require input from viewer, an important new feature of moving image art.

For filmmakers, some of the questions surrounding generative montage might be how algorithmic film affects and extends theories of traditional cinematic montage? What new kinds of juxtaposition are made available through algorithmic film? How does the pace, pattern, and variable complexity of generative montage affect ideas about pace, rhythm,tonal, over-tonal and other types of montage? Are abstract ideas created from the collisions and juxtapositions, the theses and anti-theses, of generative montage?

Observations

While too painstaking to be attempted by hand, or even with traditional non-linear editing softwares, generative montage allows for single frame, variable assembly of long patterns of image each with complex and evolving production rules. In the films noted below, the speed, complexity, and range of algorithmic assembly do seem to offer new ideas or insights into the affect of moving image montage.
The results of algorithmically edited film sequences point to varieties of montage that are startlingly fast-paced, dense and overlapping, several of which can be said to create new categories of montage, new emotional responses and, perhaps most strikingly, new intellectual abstractions.

Classic A/B scene editing is widely used in the entertainment industry as a way of allowing the camera to become informational, usually shifting from camera set up to camera set up as a way of covering dialogue scenes in a film and allowing cinematic focus to remain with the speaker. The Conversation uses the classic Hollywood A/B shot intercut with itself to create a non-linear reframing of character interaction. In this generative version, the forward motion of the story is disrupted by the rapid intercutting of similar but non-sequential frames. A term that could be used for this rapid non-sequential but familial intercutting of a frame sequence might be 'inter-linear'. While most traditional techniques of montage were designed to draw audiences deeper into the films, inter-linear montage seems to add distance to the viewer's interpretation of character. The Conversation shows how an inter-linear cut adds a kind of distance that might be compared to a Brechtian Verfremdungseffekt.

The film Shootout is also interlinear, but here the rapid pace of the edited sequence dissembles narrative not so much by disrupting the logical flow of information, but by unraveling the cinematic framing of 'bad guy' versus 'good guy'. In this example, generative montage – along with composition within the frame – creates a kind of blurring together of protagonist and antagonist, which some viewers might believe effects a kind of thematic or moral equivalency. In this context, a case where scenes are re-assembled without strong moral vectors, there is again a sense of time being suspended, of spatialization, and of distance being created, but more important seems to be the sense that, through a rapid juxtaposition, even overlapping of shots, the compositional framing of good versus evil is re-configured. This kind of 'totalizing' montage, equalizes all kinds of characters in a purely cinematic way.

Angela Ferraiolo, Three Hollywood Grammars: The Conversation (2015), USA, 7mins, 40 secs excerpt. Hi-res .tiff sequence, color/sound.

Angela Ferraiolo, Three Hollywood Grammars: Shootout (2015), USA, 7mins, 40 secs
excerpt. Hi-res .tiff sequence, color/sound.

Like The Conversation and Shootout, the montage of Chase disrupts the logical flow of information as typically enforced by standard Hollywood editing, is again disrupted creating alienation from characters and distance from story, but here, in addition, time itself, so critical to chase scenes, seems to virtually stop and stand still. As a result, the two characters, 'bad guy' and 'good guy' seem more opposed than ever. In fact, in an effect that seems the opposite of the totalizing montage of Shootout, probably because of the composition of characters and their poses within the frame, Chase seems actually to emphasize the opposition between hero and villain, in some ways possibly pushing their conflict towards a meta-realm that might be said to lie somewhere beyond or above the screen. In it's ability to call forward the 'super-story' and also to stop time, this kind of montage might be called 'meta-narrative' or 'spatial”.

In each of these films, the freedom of process allowed by computation has affected the cinematic identity of the work. While similar in production and technique, generative montage does demonstrate several new ways of thinking about juxtaposition in moving image: inter-linear which privileges the cinematic over the informational, totalizing which creates distance from the ideological values of the film narrative while drawing opposing characters closer together on some moral stage, spatial which stops time and presents sequence as a compositional event, and meta-narrative which is alienating to the degree that the audience may find itself completely outside a film whose conflicts have become pure abstractions.

Conclusions

These are just a few initial observations on generative montage. As more filmmakers adopt algorithmic and generative techniques, it is likely that the range of ideas surrounding new kinds of montage and new effects will diversify. In the future, it will be interesting to see how artists inspired by Hitchcock's observations, Vorkepich's metaphoric imagination, and Eisenstein's ideological sensitivities use generative techniques to compose a wider range of moving image effects.

Bibliography

Eisenstein, Sergei, and Richard Taylor. The Eisenstein Reader. London: British Film Institute, 1998.
Hitchcock, Alfred, and Sidney Gottlieb. Alfred Hitchcock: Interviews. Jackson: University Press of Mississippi, 2003.
Manovich, Lev. Soft Cinema: Navigating the Database. Cambridge, MA: Distributed by the MIT Press, 2005.
Vertov, Dziga, Annette Michelson, and Kevin O'Brien. Kino-eye: The Writings of Dziga Vertov. Berkeley, Ca.: University of California Press, 1984.