1 Taking Materiality Seriously
While we can observe a foregrounding of ‘non-linearity’ due to recent technical and creative developments, this is certainly not a new phenomenon. By leaving the main text to read its footnotes and follow the references, even a printed academic article embodies some of the main characteristics underlying the notion of hypertextuality and its nonlinear, or rather multilinear,1 experience. However, as we move into electronic hypertext systems2 and networked environments such as the web, these referenced materials become much better available as (multimedia) parts of the texts themselves or as hyperlinks to other online sources. As a result, informal reading practices such as skimming, browsing or task switching are increasingly stimulated.3 Furthermore, authors who produce work in these contexts might explore the materially distinct affordances to develop different styles of writing.4 Drawing towards exploration and expansiveness, hypertext served as an attempt to find forms of representation that would match the associative and inexhaustible workings of the human mind.5 Disrupting the mainstream conventions of narrative linearity, closure and coherence while still emerging out of a literary heritage, however, the novelist Robert Coover already reflected on the challenges that this form posed to writers and readers of fiction in his 1992 essay ‘The End of Books’.6
In the context of nonfiction such as documentary, journalism and academia, literary linearity has clearly left its mark on dominant conventions, not least due to the legacy of analogue media with their particular affordances and constraints. Moreover, as a strong ingredient in particularly Western renderings of the world, linear narrativity also functions as primary mental and social means for orientating ourselves and interpreting complex sensory experiences. Regina Bendix, for instance, has demonstrated how storytelling serves as a “disaster relief” to regain order out of chaos after a catastrophic event.7 These conventions, again, have shaped reading, listening and viewing expectations that can be shattered easily when disruptions occur. To find acceptance within the scientific community, research results, for instance, have to meet certain socio-historically evolved and institutionalised standards, one of them being traceability. Although, we can find some interesting, often meta-reflexive, experiments in nonfiction or “serious hypertext”8, Janet Murray also commented:
Most of what is delivered in hypertext format over the World Wide Web, both fiction and nonfiction, is merely linear writing with table-of-contents links in it. Even those documents designed explicitly for digital presentation, both fiction and nonfiction, often require too much superfluous clicking to reach a desirable destination or so much scrolling that readers forget where they are. The conventions of segmentation and navigation have not been established well enough for hypertext in general, let alone for narrative. The separation of the printed book into focused chapters was an important precondition of the modern novel; hypertext fiction is still awaiting the development of formal conventions of organization that will allow the reader/interactor to explore an encyclopedic medium without being overwhelmed.9
In the field of currently evolving documentary practices with which this essay is concerned, we can observe a similar status quo. The internet has become a site for expanding documentary beyond the scope of cinema and television broadcasting. Despite providing content on-demand, web-specific formats have developed to supplement TV programming. Accompanied by converging media conventions and the emergence of new practices and aesthetics,10 the increasing appropriation of digital technology has also led to a diverse palette of documentaries that aims to incorporate computational affordances into the ‘creative treatment of actuality’.11 However, my contention is that narrativity and linearity still prevail as guiding principles for a majority of these practices, while experiments aiming to investigate the logics of digital environments beyond the narrative paradigm remain at the margins. Based on my case study of the Korsakow System, I will demonstrate how the development of a counter-practice challenges the expectations of users, viewers and the interactive documentary community and facilitates critical reflection on the reliance of narrativity and linearity in digital documentary environments.
2 An Ethnographic Approach to Materiality
In the following paragraphs, I would like to set out to explore the emerging field of interactive documentary through the lens of a specific technology and the practices that have evolved around it. Originally developed in 2000, the first version of the so-called Korsakow System derived from German media artist and documentary maker Florian Thalhofer’s intention to test the computer’s potential for storytelling in the context of studying human computer interaction at the University of the Arts in Berlin:
What I developed was a simple structure that makes sense in terms of telling stories on a computer. It’s not the way that stories are traditionally told of course. It wasn’t my intention to make a point against traditional storytelling at all; I just didn’t come from a traditional background and no one told me how to tell a story properly, so I just came up with a way to do it using a computer.12
Since the release of his first project, The Korsakow-Syndrome – a non-linear and interactive film about alcohol (2000), Thalhofer has produced eight major Korsakow films,13 founded the Korsakow Institut14 and has continuously worked on the software’s redevelopment together with programmer David Reisch and Canadian media scholar Matt Soar.15 Due to international workshops and academic conference presentations, a number of filmmakers, artists and scholars have taken notice of Korsakow and appropriated its affordances (and constraints) for authoring media files such as image, video, sound and text assets in a way that they share multiple links.16 While ‘some of the most distinguished current productions’ in the field are ‘tailor-made solutions’ that are ‘created by whole teams of creative and technical staff’, Soar argues that accessible off-the-shelf authoring tools such as Korsakow, Klynt or Racontr17 enable filmmakers without programming skills to create interactive documentaries on a low-budget scale.18 Among this expanding range of what Lev Manovich called ‘media software’, inclusive of ‘programs that are used to create and interact with media objects and environments’,19 Korsakow occupies a special position. It is the aim of this essay to elaborate on its distinctive features and the role it plays in the discourse on interactive documentary.
How can we go about studying software? Previous scholarship primarily coming out of media studies has focused on the analysis of works authored in Korsakow. According to Sandra Gaudenzi, for instance, Thalhofer’s [LoveStoryProject]20 falls under the ‘hypertext mode’, which invites users to browse a closed database depending on the possible paths created by the authors.21 Contrasting Gaudenzi, Paolo Favero argues, ‘Korsakow is based on the principle that it is the viewers who are in charge of the construction of the narrative’.22 In his discussion on Korsakow as a ‘complex, operative techno-human ecology’,23 Adrian Miles has gone further and offered a thorough description of the software’s material specificity and its enabling of ‘generative, associative patterns to emerge amongst its parts while the work is being authored and played’.24
Keeping these theorisations in mind, my propositions stem from an ethnographic angle that seeks to provide a case study of Korsakow as part of a situated practice. Informed by a new materialist perspective, this approach allows me to examine what is distinct about the object of investigation in its participation within a specific practice by focusing on the unfolding of situated actions rather than on their hermeneutical dimension. During six months of fieldwork at the non/fictionLab – an Australian research centre located in RMIT University’s School of Media and Communication ‘exist[ing] to explore and articulate the value of new creative work’25 – my intention was to understand the ways in which a small academic community produces documentary knowledge through cultivating a creative practice with Korsakow. Building on the anthropology of practice26 and the anthropology of the senses,27 Cristina Grasseni’s concept of ‘skilled visions’28 enables me to explore the ways vision is trained, perception is oriented and understanding is structured within this particular ‘community of practice’.29
Rather than analysing ready-made works produced with Korsakow, the praxeographic method adopted here will focus on the interrelations of visual enskilment specific to the documentary apprenticeship I observed at the non/fictionLab from 2015 to 2016. Shifting between observer and participant, I attended workshops, supervision sessions and staff meetings to study ‘the role played by peer-to-peer negotiation, hierarchical relations and the management of relevant contexts, narratives and artefacts in the social construction of skilled visions’.30 I used a camera to film guided interviews and analyse routines with Korsakow. I also started authoring material in the software myself. At the same time, I shared the footage and reflections on my research blog,31 co-organized the Docuverse Symposium32 with some of my research participants and gave several presentations to facilitate transparency and discussion.
As we have established already, there are multiple ways of engaging with Korsakow’s materiality, and even within the non/fictionLab, approaches vary. These multiple versions are not solely the result of the software’s ongoing redevelopment. They also come into being through different traditional ‘prisms used in media and cinema studies […] characterized as relying on either textual, institutional or audience specific approaches’33 as well as differently ‘skilled engagements’.34 However, there is still something quite specific about Korsakow’s role in the documentary training observed at the non/fictionLab. Grouped around the current co-director Adrian Miles, the research-practitioners under study exhibited a particular view on documentary through their use of Korsakow that I will refer to as the ‘Korsakow perspective’.
What is specific about this perspective in relation to the particular environment in and people by which it is enacted? What kind of documentary knowledge are they producing in the process, and how does this relate to the wider field of contemporary documentary practices? In attempting to answer these questions, this text will probably not do without a certain degree of linear storytelling and perhaps it does not need to. Against the backdrop of a brief survey of the current climate in interactive documentary, I will turn to the stories collected in my fieldwork diary and use them as tools to render the experiential dimension of people’s encounters with Korsakow (including my own) and make the perspective(s) more graspable. Fragments of filmed material will complement the line(s) of argument and continue the dialogue with my research participants.35 Between local apprenticeship and global standards, my case study of the Korsakow System will demonstrate how the emergence of this particular Korsakow perspective as an academically situated documentary ‘schooling of the eye’ 36 challenges other, more established, documentary enskilments and related cognitive processes.37
3 New Ways to Understand the World: the Field of Interactive Documentary
Documentary is a troubled genre that encompasses a long and complicated history of diverse theories and practices. Frequently scrutinised for its ambiguous relation to reality, truth and evidence, some scholars have gone so far as to deny its very existence. Following Trinh T. Minh-ha, for instance, ‘[t]here is no such thing as documentary – whether the term designates a category of material, a genre, an approach, or a set of techniques’.38 On a more positive note, Christian Huck and Jens Kiefer pointed to the genre’s double bind that ‘it is not like fiction and it is like fiction’ as a source for both irritation and inspiration.39 In its in-between state, transgeneric and transmedia hybridisations have occurred that deliberately turn towards what Heinz B. Heller conceived as the ‘transitory’ of documentary in order to explore other, more experiential, modes for engaging with the world.40
As we progress further into the spaces of electronic and networked environments, the blurred boundaries between narrative forms and the affordances of digital media are becoming another crucial site for creative experimentation. However, reflecting on the series of database documentaries produced in the context of her ‘Labyrinth Project’ at the University of Southern California since 1997, Marsha Kinder notes that ‘interactive narrative did not begin in cyberspace. It has deep, tangled roots in an array of earlier forms—ritual, theater, poetry, novel, dance, music, radio, cinema, television, performance art, and dreams. But new digital media provoke us to redefine these two concepts— narrative and interactivity—in productive ways’.41 One approach towards investigating digital environments as tools for storytelling is to thoroughly consider their distinct characteristics. Janet Murray, for instance, already pointedly demonstrated the need for developing new narrative forms alongside the computer’s procedural, participatory, spatial and encyclopedic logic to fully exploit its potential for literary renderings of human experience.42 In the context of interactive cinema, Graheme Weinbren drew attention to the opportunities opened up by new media for rethinking the notion of narrative linearity that had been adopted by cinema from 19th century literature and has ‘continued to rule vast regions of media territory ever since’.43
As two of the early contributors to and initiators of the current academic discourse on interactive documentary (i-Docs),44 Judith Aston and Sandra Gaudenzi also emphasise these opportunities:
The focus of the i-Docs symposia was not to be about debating the merits of linear versus interactive formats, but more about understanding the new opportunities that were being opened up by the development of interactive technologies within a twenty-first century context.45
In the unfolding plentitude of different approaches to understand and shape the world through digital technology, Patricia Zimmermann and Helen De Michiel regard documentary to be ‘redefining itself as a fluid, collaborative, shape-shifting, responsive environment for encounters [they] call open space documentary’.46 From enabling users to navigate a database to generating their own content or stepping into a virtual reality, the spaces created in these works differ significantly. However, they share an increased focus on actively engaging the viewers in the documentary process. Digital interactivity47 in its different modes48 and dimensions49 appears to be at the heart of these developments, both structurally and theoretically, to reshape the relations between viewers, the documentary and the re-presented reality.
As mentioned in the introduction, Sandra Gaudenzi developed a typology of interactivity in which she draws on Thalhofer’s [LoveStoryProject]50 authored in Korsakow as an example for her ‘hypertext mode’, which follows the assumption that ‘our world is pre-determined, although full of options, and that our power lies in choosing our path, not in creating or changing such world’.51 In contrast, the ‘participatory mode’52 supposedly facilitates the co-creation of reality with the help of the viewers by ‘contributing to a constantly evolving whole, that might never be finished’.53 Depending on the mode of interactivity, accordingly, different dynamics evolve to attribute viewers with an active role in the negotiation of a certain notion of reality:
In this sense, each form of i-doc seems to negotiate reality far beyond Stella Bruzzi’s vision of documentaries as ‘performative acts whose truth comes into being only at the moment of filming’ because the ‘moment of truth’ is now also placed into the actions and decisions of the user/participant. We see this way of thinking about i-docs as offering a tool as much for the co-creation of reality as for its representation.54
Letting the viewers become participants in a dialogue rather than confronting them with an authorial and fixed point of view about the world constitutes a presumably more ‘authentic’ organising principle for re-presenting realty. While this perspective still echoes the notion of truth as the main reference point, it clearly signals a shift from representing to negotiating individual experiences. However, these performative opportunities of sense-making through interactivity are bound to the specific structures, rules and framings of the invoked spaces.55 While most of the latest productions indeed display efforts to incorporate multiple perspectives and facilitate choice or chance for user engagement and contribution, they still operate in rather fixed frameworks and linear conventions of storytelling. At the 22nd iteration of Visible Evidence – an international conference on documentary film and media, Adrian Miles critically commented on this development:
Observations about multilinearity in interactive documentary are commonplace, with a litany of celebratory arguments about empowered users, innovative narratives and radical forms. It seems though that story, as a particularly reified form of narrative, remains an unquestioned, almost invisible centre, with all the hallmarks of what we once would have called an ideology.56
Returning to Heller’s notion of exploring documentary’s experiential dimension, Bettina Frankham, for instance, has drawn on Bill Nichols’ poetic mode of documentary57 to conceive it ‘not as a replication of the world but as an aesthetic experience in its own right’.58 On that note, it would be interesting to further investigate what the essential properties of digital environments (including but not limited to interactivity) might offer documentary beyond the narrative paradigm.
4 Multiple Stories, Multiple Practices, Multiple Korsakows
Throughout the many attempts to advertise Korsakow as ‘the future of story telling’59 on his websites and during interviews, workshops and presentations,60 Thalhofer has repeatedly stressed the software’s potential for rethinking the structure of storytelling:
Korsakow is just a tool and you could use it to create all kinds of stories. You could use it to make a traditional, linear documentary if you wanted to. But at the same time, it does invite people to think about the structure of storytelling and it does encourage openness in terms of the structure of stories.61
These structures can be linear or multilinear, simple or complex; they can be built on loops and repetitions to create rhythms and patterns. If, by design, Korsakow affords such an openness, it hardly comes as a surprise that there are multiple stories, practices and versions of Korsakow.
This short video sequence is the result of a series of interviews I recorded with media scholars at RMIT University’s non/fictionLab. As we can learn from these excerpts, they have adopted Korsakow in various ways within their practice-based research. Following attempts in social studies of science to describe the fluidity of objects as they wander from one context to another,62 Korsakow becomes tangible only if we circle around its many configurations through specific perspectives. From the graphic user interface of a work produced with Korsakow, it is often difficult to understand what is going on behind its surface. When looking at Hannah Brasier’s PhD project that she is in the process of completing under the supervision of Adrian Miles at the non/fictionLab, Korsakow presented itself to me as a still (uninformed) viewer in the shape of a website offering different video clips and still images.
On the far left hand side of the screen, I see an edited video sequence of what appears to be short impressions of live music events accompanied by a women’s voice making sounds. As I move my cursor over the three still images next to it, they turn into videos, but the audio stays mute. Not sure what is expected of me, I click on the image on the far right depicting pink bunnies. Straightaway, a new interface loads presenting a new video clip on the left displaying a collection of pink things. The female voice is reading a list entitled ‘indecisive floral decisions’ including ‘pink flowers, pink ball gown, […] finding friends, losing friends, bumping into people, […] waiting, leaving walking’. With every click, the chosen clip wanders to the left and a new set of preceding images is generated. After spending about five minutes on watching and clicking, I close the browser. Without any clue of how far I have progressed within the ‘film’, I am irritated. Instead of a documentary story, I encountered fragmented video clips that appeared to play randomly.
‘This minimalism in interface design, a graphic navigation surface that is reduced to the most basic features’, is one of the aspects for Anna Wiehl that sets Korsakow films apart from other emerging interactive practices.63 At first glance, this minimalism might appear banal to people who have not been trained to use or view it. As Florian notes, ‘[…] people find it boring. They say it is a boring experience’.64 However, after looking over Hannah’s shoulder and behind the graphic user interface, I gradually begin to learn that every viewing of the ‘same’ work is a different experience altogether. What I see is different from what you see, and making sense of these generative and associative poetics requires skill, repetition and risk.
5 A maker’s Perspective: Korsakow Wants to Let Things Float
It is quiet in the big open space of the Higher Degree Research Lab at RMIT’s School of Media and Communication on this Tuesday afternoon. Everyone is sitting in front of their computers deeply absorbed in the flickering screens. Hannah’s rhythmical clicking and typing is periodically interrupted by long pauses of what seems to be nothing but staring at the monitor. Her desk is covered in books on new media, affect theory and documentary. A quote by James Benning is dangling from the shelf: ‘I have a very simple definition of an artist. The artist is someone who pays attention and reports back’. Next to a photograph of a palm tree, schematic diagrams are mounted on the wall of the little cubicle depicting the connection between ‘multi-linearity’, ‘aesthetic noticing’ and the ‘messiness of the world’. Hannah’s journal lies open in front of her. Every entry is highlighted by a red headline: ‘Art Party, Not Party’, followed by a list of what appears to be random associations: ‘indecisive floral decision, pink flowers, pink ball gown […]’…
Although clearly irritated by my presence, Hannah nevertheless follows her routine with Korsakow while I am standing behind her observing the action with my camera. With another click, she opens the application on her computer and starts importing eleven square formatted static video clips. After the program shuts down due to a system error, Hannah laughs and restarts the process in an experienced fashion until the clips appear as a list in Korsakow’s main interface. Resembling a table, the first row indicates the names of the clips that correspond with the written lists in her journal. She double-clicks on the first video file and another window opens on her screen entitled ‘SNU editor’. Out of the different attributes that she could additionally attach here to the video clip depicting ‘pink flowers’, she only turns her attention to two. At first, she types the number ‘1’ into the text fields named ‘IN-Keywords’ and ‘OUT-Keywords’. After that, she clicks on the drop-down menu ‘Lives’ and assigns ‘’…
Puzzled by Hannah’s process with Korsakow, I ask her to explain what she is doing:
One of Korsakow’s fundamental principles is working with keywords to choreograph the multiple relations that can occur among the media assets imported into the system. Keywords are strings of texts that can be content-driven, abstract and multiple. There are two different types of keywords: IN-Keywords specify the respective clip while OUT-Keywords define possible relations to other clips. Thus, ‘keywording’ can be understood as a form of tagging or rule making that creates multiple simultaneous links between different parts. When Hannah exports her project into a web-ready HTML-package, Korsakow’s runtime engine performs searches based on the given keywords. After assigning each video clip the same IN- and OUT-Keyword (1), consequently, the clips play randomly as they are all connected in equal measure.
Observing the still images displayed on a black interface in her web-browser, Hannah moves the cursor over the first and it comes to life: a short, edited sequence of various blue skies starts playing. She moves over to the next image. Different forms of shadows falling onto grey walls join in. Hannah watches the two clips carefully as they play simultaneously, and her eyes are fixed on the screen. Deeply absorbed, nothing appears to be happening for a long time. Suddenly, she grabs her black pen and scribbles a note in her book: ‘light’. A little bit later, another one follows: ‘colour’. She returns to the Korsakow program and double-clicks on the first video file until the SNU editor opens once again. She deletes the number ‘1’ she had previously typed into the text fields and replaces it with ‘light’ as the IN-Keyword and ‘colour’ as the OUT-Keyword. Continuing in the same manner, Hannah systematically returns to each of her clips. Sometimes she drags ‘light’ from the keyword database and drops it in the IN-Keyword and places ‘colour’ in the OUT-Keyword field of the SNU editor; sometimes vice versa. The project is exported again and the clips appear in the internet browser, though now in a slightly different arrangement leading to more clicking, observing and note taking …
Hannah will repeat this process numerous times. In fact, ‘the programmatic dance that is Korsakow’65 has recently begun to inspire how her keyword-design will evolve throughout her film project and which video clips Hannah is going to shoot next.
Why is Hannah doing what she is doing? For Hannah, Korsakow’s keywording principle fits in well with her creative research practices of list-making and aesthetic noticing66 to build associations around a particular observation from her everyday life: ‘[…] keywording is kind of listing; it’s associating a clip with a certain amount of keywords and that is kind of building this database of lists’.67 Within the recursive process she is describing, the keywords that are at first connecting the collected clips randomly reveal aspects within the material that she was not aware of at the moment of shooting. By discovering the relationships between the clips, she gradually elaborates the keyword-design until different thematic clusters begin to emerge. However, Korsakow’s keywording principle does not only inform the ‘editing’ process68:
I start by simply taking my phone on walks, capturing anything that seems to distinguish itself from other things. I film with the Vine video app on my phone because it restrains a filmed moment to 6 seconds. […]From this process of simply wandering and noticing I collect lots of 6 second pieces of footage that I then watch after about a month or so […] I then start a second phase of noticing by watching each piece of footage I have collected thus far. When I watch these Vines I write lists of everything that I notice in each clip. […] I am then expanding each clip into all of its components. I then use these written lists to prompt further filming and noticing, building quite a large collection of Vine videos. I then start editing together sequences that bring together similar things […] and it is these sequences that become my SNUs, or what I call clips, to import into Korsakow.69
Hannah has moved away from professional equipment to the built-in camera of her mobile phone. For capturing snapshots of the everyday, she makes use of the smartphone’s portability and shoots videos within the social media application Vine70 to explore the world around her more freely but at the same time under certain restrictions. Thus, different levels of noticing through listing (or keywording) shape the content and form of her entire (potentially never-ending) project. As she confirms, ‘Korsakow influences each step of how I make my multilinear documentaries, as I am particularly interested in how we can make documentaries with Korsakow as opposed to translating already conceived of projects through Korsakow’.71
Another function that Hannah has attributed to the individual videos within the editor is the number of lives that refers to the amount of times each clip can be viewed. From one to five or infinite, Hannah chooses ‘∞’. Thus, the edited sequence of blue skies will appear as often as the generative system finds a connection from the currently playing clip’s OUT-Keyword to the blue skies’ IN-Keyword. If Hannah had chosen ‘1’, each clip could have only been watched once before it disappears from the K-film. Since it is Hannah’s objective to find possible connections amongst her material, it is crucial for her practice to allow the occurrence of loops and repetitions so that rhythms and patterns can emerge.
In her practice-based PhD project on ‘Negotiating Mess – Towards a Multilinear Engagement with Complexity in Interactive Documentary’,72 Hannah follows John Law, Ross Gibson and Lev Manovich’s conceptualisation of the world as messy to underline its ever-growing complexity.73 In our joint presentation at the Docuverse Symposium,74 citing Matt Soar, Hannah argues that in contrast to linear documentaries, which ‘generally necessitate the corralling of footage into a singular narrative arc with a beginning, middle, and end, involving a compelling main character or characters, a problem to be solved or an adversity to be overcome, topped off with a memorable denouement’,75 Korsakow’s variability and openness would allow for engaging with complexity by arranging audiovisual material through the keywording principle in a multilinear way. In our interview, Hannah told me about the sense of freedom and surprise she associates with her first encounter with Korsakow. The software did not require her to craft a story; in fact, it proved to be quite difficult to do so. On the contrary, as a rule-maker and choreographer in Korsakow, Hannah feels liberated from the rigid structures of her documentary training. Now empowered to define the terms of what she is making and to refine her creative practice in the process of her PhD, Korsakow has become a constitutive participant for making multilinear documentaries that, from her point of view, have a better connection to the world:
6 A Teacher’s Perspective: Coping with Openness
Gathering around her laptop computer, I film Hannah as she presents the sequences she has been editing together from a collection of Vine video clips for her next Korsakow project to Adrian Miles and Wil Polson during a supervision session in December 2015. Each sequence represents a list of clips that expand on one idea or theme taken from her written lists, she explains. We watch the first video, a collection of pink things: several close-ups of pinks flowers are followed by drawings of pink bunnies, pink phone booths and pink fabric. After a series of other videos, Adrian and Wil comment on Hannah’s practice of list making and noticing in relation to the way she has edited the sequences. Was it one moment she had noticed or rather a study of one subject throughout different situations? Why not think about the rhythms of a Korsakow film in order to make the ‘Vine-lists’ less flat and more variable? Not everything you notice has an equal impact; not every word in a list is of the same length. The world is messy, they agree …
What ‘works well’ and ‘looks good’ in Korsakow is not just dependent on Hannah’s own preferences of making poetic and personal documentaries that let the viewer wander within their loose organisation.76 Hannah’s visual practice is highly influenced by Korsakow’s materiality just as much as it is embedded in the social dimension of the local documentary apprenticeship. She is based in the non/fictionLab and supervised by Adrian Miles, one of Korsakow’s strongest academic advocates who uses the software for theorising and teaching interactive documentary. As a research centre situated within RMIT University’s School of Media and Communication, the non/fictionLab converges a wide spectrum of interdisciplinary approaches to fiction and nonfiction media-making under a single roof. Within the documentary strand, through practice-based research with film, photography, performance, poetry and other forms of writing, different tools are employed to explore different ways of engaging with the world. The specific research culture and creative atmosphere of the lab, then, has a strong impact on Hannah’s work. Within the broader context of new media nonfiction, diverging from established norms and conventions is not only accepted but also encouraged by this community of research-practitioners.
In response to my request, Adrian has organised a ‘Korsakow Interactive Video Workshop’. Shortly before the Christmas break, six students, mostly associated with the non/fictionLab, have turned up on this warm Wednesday midday to make and discuss interactive documentary works with Korsakow, among them is Hannah Brasier. After the participants have made some remarks on their motivation for attending, the day kicks off with a thirty-minute-long presentation by Adrian, who introduces Korsakow and its main principles. His first slide is projected onto the wall of the Honour’s Lab. Under the headline ‘what it is for’, three points are listed: ‘making associative & generative interactive video works that use a web browser’, ‘less useful for menu based “informational” interactive video’ and ‘well suited to documentary’. After explaining the keyword principle and the recursive workflow, Adrian highlights the meaning of lives for creating rhythmic structures with Korsakow …
In this example, Adrian stresses the software’s generativity and compares making an interactive documentary in Korsakow with the mode of composition in music or poetry. In order to find the rhythms of the work, he encourages the students to make use of Korsakow’s ‘hypertextuality’: its ability to let an individual work within a specific material environment that supports aleatory methods. By assigning every clip at least two lives, for instance, loops and repetitions would help to recognise the following:
[…] you do not ‘return’ to the same clip as its repetition means it is and cannot be identical to its first appearance. The structure, therefore, of this return is musical more than narrativist, it is verse and chorus rather than the literary’s normative embrace of time’s arrow, and what clips come to mean change because of, not in spite of, their reocccurence.77
By bringing his hypertext research back into a new media studies program and through the lens of new materialist theories, Adrian’s specific approach to Korsakow is informed by his notion of interactive documentary as ‘the enabling, discovery and choreography of […] relations’.78 As a result, he has conceptualised Korsakow as a ‘deeply entangled interconnected system’79 that could facilitate a ‘proactive engagement with the world’80 as an ‘implicitly multi-causal, messy’ place where ‘multiple agents with multiple facets […] interact with each other’.81 Repetition, then, would allow viewers to realise that their decisions change the shape of the K-film. If only people could ‘cope with its openness’:
7 Story as Default
3rd of March 2016. It is the evening of day two at the fourth i-Docs symposium. Bristol’s weather feels like a shock compared to summerly Melbourne. Yesterday, Florian gave his keynote on Korsakow, which was not exactly on Korsakow but rather on the overall issue of linear storytelling. I still don’t understand why he was invited, again. Adrian told me he thinks the initial response to Korsakow was dismissive because people misunderstood what it does and that the regained interest was a response to his and Matt’s publications advocating Korsakow and explaining what it is and how it works. The first day was indeed dominated by critical discussions surrounding the all-pervasive need to storify, as if it was intentionally programmed that way. Alisa Lebow’s keynote on her project ‘Filming Revolution’ already addressed the idea of moving away from storytelling when representing research outcomes in interactive spaces. Observing Florian’s reaction in the audience and informally chatting to him over dinner last night, I could tell that he was thrilled by the idea that people finally picked up on what he had been preaching for almost a decade. Adrian, however, displayed a more critical reading of what was going on at the symposium …
Why is there so much resistance? Why do people feel that need to storify their interactive projects even though research participants and even makers themselves claim that they are not stories? What we can derive from the co-constitution of Korsakow’s specific materiality and the different stories surrounding it, is that what is referred to as Korsakow’s potential in the experimental space of new media practitioner-researchers at the non/fictionLab (and within the wider Korsakow community) dissolves in disappointment and resistance to embrace its openness. There is something disconcerting about Korsakow in that it challenges highly conventionalised linear schoolings of the eye which permeate the discourse of interactive documentary but also film production, reception and related cognitive processes more broadly.
While teaching Korsakow in the context of the local Master programme in Cultural Anthropology with an emphasis on Visual Anthropology (CVA) at Göttingen University,82 I was confronted with exactly these challenges. During our first session in which I asked the students to individually engage with Thalhofer’s Planet Galata83 and reflect on their experience, the majority of them expressed frustration and stress. They felt overwhelmed by the K-film’s information (over)supply, small-sized and multimodal arrangement, lack of structure, tension and closure. ‘The collective experience of cinema, eating popcorn and leaning backwards is what makes film special to me’, commented one of the students. Another one noted that the need for active selection was a distraction from the individual clips and that there was no real agency in choosing which protagonists to follow. Among other things, their sense of disappointment and reluctance to enjoy the unusual experience was clearly stemming from their background in and knowledge of linear documentary. Thus, the criteria they utilised to view and evaluate the K-film turned out to be misleading. The discussion also revealed that it was perhaps the viewers’ missing of literacy for understanding the format: How do we watch these films? How can we evaluate and talk about them? Can we refer to them as ‘films’ at all?
8 Narrativity versus Messiness: Different Schoolings of the Documentary Eye
While it is important to acknowledge narratives as a ‘means of patterning and interpreting the meaning of all sensory input and ‘objects of knowledge’’,84 it is of equal importance to reflect critically on the central role stories play as ‘colonisers’ of the world, and, accordingly, as interactive documentary.85 This is not to deny the range of narrative styles and structures that emerged from creative experimentation. However, the distinct characteristics of digital environments also lend themselves to non-literary creation. Although Thalhofer originally conceived Korsakow as a tool to explore storytelling computationally, my case study of the Korsakow System has demonstrated how the openness of the software’s design has invited documentary research-practitioners to rethink their practices beyond narratives:
The generative, procedural possibilities that interactive documentary offers have affinities to the world that make it distinct from those that story and narrative and representation offers. It is not that we should not use stories in interactive documentary, but that we are colonisers of interactive documentary via story. I am not sure I know how to say this simply or clearly, however, if the world is made of things with agency in their own right, and, if particular ways of making procedural, generative multilinear works also allow things to retain their agency, then we have a possible nonfiction practice and form that adopts, at least to some extent, the points of view of the world.86
The recursive process with Korsakow observed at the non/fictionLab allows for a more in-depth and expanded analysis of the complex relations that can occur among the captured aspects of the world. Thus, a skilled engagement with Korsakow’s generative algorithm lets its users consider the different facets of their material in various constellations through repeated noticing, clicking and keywording.
As Grasseni remarks, the construction of knowledge is only rendered meaningful if certain local, situated and contextual conditions apply.87 In order to cope with Korsakow’s openness, it requires specific training and an environment that facilitate a multisensory attunement to the complexity of the world through a mode of documentary-making in which surrendering authorial control becomes a necessity and redundancy is welcomed. By closely analysing Hannah Brasier’s documentary practice in its socio-material constellation of knowledge production, I was able to emphasise the importance of specific local ‘exercises of vision’88 that often involve a certain degree of defamiliarisation with other institutionalised trainings and their embedded ideologies. Thus, what is defined as ‘good’ work in this setting is highly dependent on and structured by the respective training. Moreover, it might be challenged by standard or default professional conventions as well as by socio-cultural hegemonies.
In line with documentary’s experiential qualities, the skilled visions emerging around Korsakow are but one example for taking computational materiality seriously and exploring its potential for performing patterns of experience that defy narrative renderings. In this sense, Korsakow occupies a special role among other authoring tools for interactive documentary because it wants the generative algorithm to retain and demonstrate its agency. As a counter-practice to the dominant discourse in interactive documentary, the Korsakow schooling developed in the non/fictionLab can help us to reimagine what documentary could become in multilinear digital environments. Anna Wiehl, for instance, has considered Korsakow documentaries ‘as affective tools for thought, as poetic miniatures zooming into the very small details of everyday life in a contemplative way’,89 and Sharon Daniel has argued for mapping sites of social dissensus through interface design and polyvocality as a counter-strategy to narrative persuasion.90
However, it is not my aim to favour generative multilinearity over narrative forms of representation. Storytelling holds specific benefits for nonfiction practices as an aid for orientation, order and emotional engagement. If Korsakow allows for a dwelling in a liminal, somewhat disorderly state and if learning how to use Korsakow is crucial to understanding what it does, can we consider it a useful tool for representation? Where does it leave the viewers if I ask them to become practitioners themselves? On a more analytical level, Korsakow certainly offers an interesting proposition for the making of documentary knowledge that deserves further attention. Whether or not it will find acceptance within a wider community of practitioners and audiences remains to be seen.
Franziska Weidle is a PhD candidate at Göttingen University’s Department of Cultural Anthropology, Germany and a research fellow at the Research Training Group "Literature and Dissemination of Literature in the Digital Age". She is interested in exhibition, installation, filmmaking and photography as forms of documentary and ethnographic knowledge production and dissemination. Since commencing her PhD on digital technology and documentary practices in 2014, Franziska has presented her work at the Digital Cultures Research Centre, the European Association of Social Anthropologists and Visible Evidence. From 2015 to 2016, she was a visiting scholar at RMIT University’s non/fictionLab where she co-founded the Docuverse Group.
1 As I learned from my conversations with Adrian Miles, the concept of multilinearity emerged from terminological differentiations in hypertext theory to acknowledge that linearity still played a role in the forming of sequences. However, how these sequences are formed is subject to variation. See: http://vogmae.net.au/vlog/2016/02/multilinearity-notes-on-terms/ [Accessed: 9 December 2016].↑
2 Cf. HyperCard, the Intermedia system developed at Brown University or Eastgate’s Storyspace.↑
3 For a recent examination of how digital technology is affecting reading practices, cf. Naomi S. Baron, Words Onscreen: The Fate of Reading in a Digital World, Oxford University Press, 2015.↑
4 The literary hypertext experiments coming out of the ‘Eastgate School’ in the late 1980s and early 1990s in reaction to poststructuralist and postmodern theory are one example. Cf. Scott Retterg, ‘The American Hypertext Novel, and Whatever Became of It?,’ Hartmut Koenitz and Gabriele Ferri et al., eds, Interactive Digital Narrative: History, Theory and Practice, Routledge, 2015, pp. 22–35.↑
5 Cf. Janet Murray, Hamlet and the Holodeck, The MIT Press, 1998, pp. 90–94.↑
6 See: http://www.nytimes.com/books/98/09/27/specials/coover-end.html [Accessed: 9 December 2016].↑
7 Regina Bendix, ‘Reflections on Earthquake Narratives’, Western Folklore 49, 4, 1990, 331–347.↑
8 As one of the most prominent publishers in the early days of hypertext and developer of the hypertext system Storyspace, Eastgate promotes itself in Robert Coover’s words as “the primary source for serious hypertext”. See: http://www.eastgate.com/ [Accessed: Dec 9, 2016]. For some examples of academic hypertexts cf. David Kolb, Socrates in the Labyrinth: Hypertext, Argument, Philosophy, Eastgate Systems, 1994; Adrian Miles, ‘Realism and a General Economy of the Link’, Currents in Electronic Literacy, 5, Fall, 2001; or Roderick Coover, Cultures in Webs: Working in hypermedia with the documentary image, Eastgate Systems, CD-ROM, 2003.↑
9 Murray 1998, p. 87.↑
10 For one of the seminal works that led to an increasing critical interrogation of cultural-digital reconfigurations, cf. Henry Jenkins, Convergence Culture: Where Old and New Media Collide, NYU Press, 2006.↑
11 John Grierson, ‘The Documentary Producer,’ Cinema Quarterly 2, 1, 1933, 8. http://archive.org/details/cinema02gdro [Accessed: 9 December 2016]. For a historical survey of the critical discussions on Grierson’s definition of documentary, cf. Brian Winston, Claiming the Real: The Griersonian Documentary and Its Legitimations, BFI, 1995.↑
12 Kate Nash, ‘An Interview with Florian Thalhofer, Media Artist and Documentary Maker,’ in Kate Nash and Craig Hight et al., eds, New Documentary Ecologies. Emerging Platforms, Practices and Discourses, Palgrave Macmillan, 2014, p. 192.↑
13 Cf. Florian Thalhofer and Mahmoud Hamdy, 7 Sons, 2003; Florian Thalhofer, [LoveStoryProject], 2007; Florian Thalhofer and Berke Bas, Planet Galata: A Bridge in Instanbul, 2010, or Florian Thalhofer, Geld.GR. Money and the Greeks, 2013.↑
15 Until recently, Korsakow projects required Adobe’s Flash Player to run. With Apple’s decision not to support Flash on their mobile devices and Google’s decision to block Flash on their Chrome browser by default, Korsakow users were anticipating the software’s move to HTML5. Released in October 2016, Korsakow 6 is now exporting to HTML5.↑
18 Matt Soar, ‘Making (with) the Korsakow System: Database Documentaries as Articulation and Assemblage,’ in Kate Nash and Craig Hight et al., eds, New Documentary Ecologies. Emerging Platforms, Practices and Discourses, Palgrave Macmillan, 2014, pp. 154–157.↑
19 Lev Manovich, Software Takes Command, Bloomsbury, 2013, p. 38.↑
21 Sandra Gaudenzi, The Living Documentary: From Representing Reality to Co-creating Reality in Digital Interactive Documentary, Doctoral thesis, Goldsmiths, University of London, 2013, p. 101.↑
22 Paolo Favero, ‘Getting Our Hands Dirty (Again): Interactive Documentaries and the Meaning of Images in the Digital Age,’ Journal of Material Culture, 18, 3, 264.↑
23 Adrian Miles, ‘Materialism and Interactive Documentary: Sketch Notes,’ Studies in Documentary Film, 8, 3, 2014, 207.↑
24 Ibid., p. 214.↑
26 Cf. Seth Chaiklin and Jean Lave, eds, Understanding Practice: Perspectives on Activity and Context, Cambridge University Press, 1993; Theodore R. Schatzki and Karin Knorr Cetina et al., eds, The Practice Turn in Contemporary Theory, Routledge, 2001.↑
27 Cf. Lydia Arantes and Lisa Rieger, eds, Ethnographie der Sinne, transcript, 2014; Sarah Pink, Doing Sensory Ethnography, Sage, 2009; Regina Bendix, ‘Introduction: Ear to Ear, Nose to Nose, Skin to Skin: The Senses in Comparative Ethnographic Perspective,‘ Etnofoor, 18, 1, 2005, 3–14; David Howes, Sensual Relations. Engaging the Senses in Culture and Social Theory, The University of Michigan Press, 2003.↑
28 Cristina Grasseni, Skilled Visions. Between Apprenticeship and Standards, Berghahn Books, 2007.↑
29 ‘Community of practice’ is a concept that was first developed by cognitive anthropologist Jean Lave and educational theorist Etienne Wenger to describe how members of an interest group develop a shared knowledge practice through a common repertoire of resources, tools, experiences and stories (Cf. Jean Lave and Etienne Wenger, Situated Learning. Legitimate Peripheral Participation, Cambridge University Press, 1991.).↑
30 Grasseni 2007, 4.↑
32 See: http://nonfictionlab.net.au/docuverse-a-symposium-for-expanded-documentary-practices/ [Accessed: 1 July 2016].↑
33 Miles 2014, 207.↑
35 The participants have agreed to be filmed and stated that the material could be published for research purposes as part of my PhD project.↑
36 Cristina Grasseni, ‘Skilled Visions. Toward an Ecology of Visual Inscriptions,’ in Marcus Banks and Jay Ruby, eds, Made to Be Seen. Perspectives on the History of Visual Anthropology, University of Chicago Press, 2011, p. 20.↑
37 To stress the importance of artefacts and their context for mediating skill, Grasseni refers to anthropological research on cognition and its development towards a study of situated action (Grasseni 2007, pp. 5–6).↑
38 Trinh T. Minh-ha, ‘Documentary Is/Not a Name,’ October, 52, Spring, 1990, 76.↑
39 Christian Huck and Jens Kiefer, ‘Documentary Films and the Creative Treatment of Actuality,’ Anglistik: International Journal of English Studies, 18, 2, 2007, 103.↑
40 Heinz B. Heller, ‘Dokumentarfilm als transitorisches Genre,’ in Ursula Keitz and Kay Hoffmann Kay, eds, Die Einübung des dokumentarischen Blicks. Fiction Film und Non Fiction Film zwischen Wahrheitsanspruch und expressiver Sachlichkeit 1895–1945, Schüren Verlag, 2001, p. 24.↑
41 Marsha Kinder, ‘Honoring the Past and Creating the Future in Cyberspace: New Technologies and Cultural Specificity,’ The Contemporary Pacific, 15, 1, Spring, 2003, 97.↑
42 Murray 1998, pp. 71–93.↑
45 Judith Aston and Sandra Gaudenzi, ‘Interactive Documentary. Setting the Field,’ Studies in Documentary Film, 6, 2, 2012, 129, emphasis added.↑
46 Helen De Michiel and Patricia Zimmerman, ‘Documentary as Open Space,’ in Brian Winston, ed, The Documentary Film Book, BFI, 2013, p. 355.↑
47 Due to the limiting scope of a journal paper, I am not going to offer a critical account for or a definition of ‘interactivity’ at this point. Accordingly, I will adhere to an outline of the leading scholars in the field and their conceptualisations. For a historical survey of the concept’s genesis in the studies of Human Computer Interaction and its adoption into literary studies, cf. Hyon-Joo Yoo, Text, Hypertext, Hypermedia: Ästhetische Möglichkeiten der digitalen Literatur mittels Intertextualität, Interaktivität und Intermedialität, Königshausen & Neumann, 2007, 74–110. Within the documentary discourse, cf. Kate Nash, ‘Clicking on the World: Documentary Representation and Interactivity,’ in Kate Nash and Craig Hight et al., eds, New Documentary Ecologies. Emerging Platforms, Practices and Discourses, Palgrave Macmillan, 2014, pp. 50–66.↑
48 Gaudenzi 2013, pp. 39–67; 93–240.↑
49 Nash 2014, pp. 52–60.↑
51 Gaudenzi 2013, p. 53.↑
52 Ibid., pp. 54–62; 176–240.↑
53 Ibid., p. 62.↑
54 Aston/Gaudenzi 2012, 128; quoting Stella Bruzzi Stella Bruzzi, New Documentary: A Critical Introduction, Routledge, 2000, p. 7.↑
55 As audience studies indicate, the anticipated engagement in interactive documentary is difficult to measure and viewers often lack skill or knowledge in order to play their expected part. Kate Nash has conducted several case studies in which she explores user engagement in interactive documentary with varying results (cf. Kate Nash, ‘Strategies of interaction, questions of meaning: an audience study of the NFBs Bear 71’, Studies in Documentary Film, 8, 3, 2014, 221–234). In response to Jon Dovey’s keynote at the i-Docs symposium 2016, she also highlighted that „engagement“ needed to be further differentiated to analyse what viewers actually do and to what extent they engage with narrative, ludic and other elements of the documentary.↑
57 Bill Nichols, Introduction to Documentary, 2nd Ed., Indiana University Press, 2010, pp. 142–171.↑
58 Bettina Frankham, ‘Space-time-engagement,’ in Adrian Miles, ed, The Aesthetics of Documentary Interactivity. A Pamphlet That Emerged from a Curated Panel at Visible Evidence XXII, Toronto, 2015, 2016, pp. 31–33.↑
60 Read more on and listen to Florian’s stories of Korsakow here: http://thalhofer.com/news/ [Accessed: 9 December 2016]; here: http://korsakow.tv/category/podcast/ [Accessed: 1 July 2016] or here: http://korsakow.tv/category/videos/ [Accessed: 1 July 2016].↑
61 Thalhofer in Nash 2014, p. 192, emphasis added.↑
62 Cf. Marianne de Laet and Annemarie Mol’s description of a water pumping device as a fluid technology. Marianne de Laet and Annemarie Mol, ‘The Zimbabwe Bush Pump: Mechanics of a Fluid Technology,’ Social Studies of Science, 30, 2, 2000, 225–263.↑
63 Anna Wiehl, ‘Database Aesthetics, Modular Storytelling, and the Intimate Small Worlds of Korsakow Documentaries,’ Necsus: European Journal of Media Studies, Spring, 2016.↑
64 Thalhofer in Nash 2014, p. 195.↑
65 Miles 2014, 214.↑
66 In an email-interview, she explains that she has developed this method of listing to notice following John Mason, Researching Your Own Practice: The Discipline of Noticing, RoutledgeFalmer, 2002; and Ian Bogost, Alien Phenomenology. Or What It’s Like to Be a Thing, University of Minnesota Press, 2012: ‘Put together in a practice of making K-Films noticing allows me to capture things, while listing allows these things to remain self-contained’ (Accessed: 1 July 2016).↑
67 Hannah Brasier, First Interview, 13 November 2015.↑
68 Matt Soar has identified three different stages of editing when working with Korsakow: (1) selecting and cutting raw material into sequences that become SNU’s in Korsakow, algorithmic editing or ‘SNUifying’ (assigning keywords, lives and other attributes in the SNU editor) and (3) viewing, i.e. selecting SNU’s and ‘thereby creating a final, non-definitive version of the film in that specific encounter’. (Soar 2014, pp. 162–163.)↑
69 Email interview with Hannah Brasier, 1 July 2016.↑
71 Email interview with Hannah Brasier, 1 July 2016.↑
73 Cf. John Law, After Method: Mess in Social Science Research, Routledge, 2004, p. 15; Ross Gibson, ‘The Known World,’ Text, 8, 2010, 7; Lev Manovich, The Language of New Media, MIT Press, 2001, p. 219.↑
75 Soar 2014, p. 167.↑
76 Hannah Brasier, First Interview, 13 November 2015.↑
77 Miles 2014, 218.↑
78 Adrian Miles, ‘I’m Not a Story,’ i-Docs Symposium, Bristol, 4 March 2016.↑
79 Adrian Miles, First Interview, 26 October 2015.↑
80 Grasseni 2011, p. 19.↑
81 Adrian Miles, First Interview, 26 October 2015.↑
82 Informed by a long tradition in making and theorising ethnographic film in Göttingen, CVA predominantly aims at using audio-visual material for representing anthropological knowledge. Key characteristics include clearly separated field phases, continued sequential editing and the crafting of a narrative dramaturgy.↑
84 Kinder 2003, p. 97.↑
85 Miles 2015.↑
86 Ibid., p. 3.↑
87 Grasseni 2007, p. 11.↑
88 Grasseni 2011, p. 22.↑
89 Wiehl, 2016.↑
90 Sharon Daniel, ‘On Politics and Aesthetics: A Case Study of “Public Secrets” and “Blood Sugar”,’ Studies in Documentary Film, 6, 2, 2012, 215–227.↑