Screen production enquiry is adopted as a creative practice research to investigate the design process of Table for Two - a multi-genre parallel interactive narrative in virtual reality. Creative practice research as an enquiry conducted to shape, document, theorize and contextualize an artwork. Considering a film as an artwork, filmmakers as researchers often use the methodology of practice as an expression and evidence of their research findings. Early novel examples of screen production enquiries can be found in the montage theory and the experiments by Normal McLaren. Cinematic virtual reality is an emerging medium that has challenged filmmakers in recent years with storytelling for 360-degree spatial experience .In cinematic virtual reality, the viewer can step inside the screen border and immerse themselves further intoa360-degree cinematic experience. In this study, the viewer is addressed as a navigator due to their active participation in a cinematic virtual reality experience. Several research on virtual reality aim to guide navigators to look at intended areas in the virtual reality environment; however, it is argued that this aspect limits the full potential of virtual reality. The active role of the navigator in cinematic virtual reality along with the freedom to choose their own viewing direction is further explored with parallel interactive narratives in virtual reality. Parallel interactive narratives in virtual reality encourage the navigator to teleport from one narrative to another through gaze selection of pinpoint areas in the virtual environment. As the navigator teleports, the remaining narratives continue to unfold in parallel, irrespective of the navigator’s presence. In parallel interactive narratives in virtual reality, the director decides the story world, whereas the navigator customizes the sequence of events in the story world as they both co-create the experience. This process of co-creation is further discussed with a creative arte fact Table for Two, a one-shot parallel interactive narrative in virtual reality that represents specific genres of romance, supernatural fiction, and drama in a café. These three genres are inter-connected through the diverse characters and events occurring in the café. This experience is designed to encourage the navigator to engage in multiple viewings of the same experience to fix the pieces of a narrative puzzle. This article discusses and reflects upon the significance of Screen Production Enquiry as a method to generate novel insights from the production process of Table for Two. This article aims to encourage more filmmakers to reflect upon their production processes through the lens of a practice-based researcher.
Table for Two(figure 1) is a one-shot Parallel Interactive Narrative in Virtual Reality (PIN VR) which is shot simultaneously with three 360-degree cameras. Cinematic Virtual Reality (CVR) is one of the latest technologies that has challenged filmmakers to emerge with new narrative methods for designing such experiences. The advantage of this medium is the 360-degree space wherein the viewer can step inside the borders of a conventional cinema screen. One of the foremost concerns in designing for CVR is to ensure the viewer looks towards the action areas in the 360-degree virtual space. To address this challenge, Sheikh et al (2016) and Pillai et al (2017) propose CVR grammar to orient the viewers towards intended areas through lighting techniques, ambisonics sound cues, 360-camera distance and editing techniques (Sheikh et al., 2016;Pillai et al., 2017). However, Gödde et al. (2018) argue that forcing a viewer to look at intended areas limits the full potential of CVR (Gödde et al., 2018). This argument is driving motivation for defining the concept of PIN VR. Studies on the increase of interaction in cinema and navigation in VR experiences discuss the role of the viewer transforming into a navigator(Daly, 2008;Friedman et al., 2007;Lo & Lai, 2023). Due to the immersive and interactive role expected in PIN VR, the viewer has been addressed as a navigator. In PIN VR, each navigator has the freedom to look wherever they desire as they can teleport from one narrative location to another by selecting hotspots in the virtual environment. As they teleport, each narrative unfolds irrespective of the navigator’s presence or absence. To further explain PIN VR, the filmmaker as a researcher has developed an experience titled Table for Two. Table for Two experiments with multiple genres such as romance, supernatural fiction, and drama to experiment with diverse narratives taking place simultaneously in a café. The navigator of this experience is encouraged to teleport from one table to the other to shift their perspective on the narrative. Within the stipulated duration of ten minutes, the navigator can select which genre to view, however other genres continue to unfold simultaneously irrespective of the navigator’s presence or absence. The parallel stories in Table for Two are scripted with interconnections that are designed to encourage the navigator to have multiple views. The PIN VR experience Table for Two is an
outcome of screen production enquiry as a practice-based research method. This article highlights the significance of practice-based research in unfolding a VR narrative practitioner’s investigation.
According to Candy (2006) practice-based research contributes to knowledge through creative outcomes such as design arte facts, films, music, digital media, performances, exhibitions, etc. (Candy, 2006). Practiced- based research is suggested to be carried out by a practitioner and the significance of the claims are mostly described in words, with reference to the outcomes. In the context of Table for Two, the outcome is a PIN VR experience, and the researcher is a filmmaker. The production design of Table for Two is utilized to reflect upon the creative decisions of the filmmaker. This article aims to address the effectiveness of screen production enquiry for gaining insights from the filmmaker perspective of their own practice. This research encourages more creative practitioners to investigate their creative work so that the creative art community can benefit from their practice-based perspective.
The impact of screen production enquiry for filmmaking
Spicer (2015) mentions the lack of support for artistic practice as academic research endeavors; however, the rapid increase of creative practice research has significantly contributed towards understanding the significance of visual communication and the creative processes (Spicer, 2015). Likewise, the creative process of filmmaking requires different perspectives and investigations. A film practioner as researcher helps build discipline specific knowledge about filmmaking practice and includes approaches that strenghten the insider’s perspective that a filmmaking researcher offers. In Australia, filmmaking research is addressed as ‘Screen Producton Enquiry’ (Kerrigan & Callaghan, 2016). Kerrigan, S. and Callaghan, J (2016) state that screen production enquiry is an iterative process of practice and reflection by a researcher who is a screen practitioner, in which the theorectical perspective informs the overall research(Kerrigan et al., 2015).Richardson et al. (2005) describes it as a condition of possibility for producing different knowledge and producing knowledge differently(Richardson et al., 2005). Dewey (2005) mentions that thinking must happen throughout the creative process as reflection on the making is essential to processing (Dewey, 2005). What if one’s practice can be a research tool to think, to know, and to discover? Filmmaking has often placed greater value on the final arte fact rather than the creation process. Often, the creative artifact emerges as something different from goals that the creator had originally envisioned. This article argues that a unique understanding of knowledge can be unfolded from the creative process. This method of filmmakers using their practice to inquire into new ideas and developments is apparent throughout history. Prominent examples are observed in the establishment of the montage theory(Kuleshov, 1910), experimental films of Norman McLaren(Dobson, 1994).
A significant contribution in experimenting with cinematic narratives is observed in the development of the montage theory by Lev Kuleshov. In the infancy of film, editing was not considered a part of filmmaking. Film productions were staged like a theatre narrative, where everything was recorded in a single long take. This can be observed in the Lumière brothers’ work in early cinema such as Workers Leaving The Lumière Factory(L. Lumière, 1895) andL'arrivéed'un train engare de LaCiotat(A. Lumière & L. Lumière, 1896).As cinema evolved, new ideas for storytelling emerged, leading to the montage theory. Amidst the socio-political scenario of the early 20th century, it was difficult to find film stock in Russia due to which, film enthusiasts studied rather than created films (Heckmann, 2020). The Moscow Film School, argued to be one of the first film schools, was founded in 1919 during the Russian Revolution. Lev Kuleshov was one of the professors who performed a series of experiments by editing footage from different sources into a whole sequence that
created an impression of continuity, thereby forming the basis of the montage. The cinematic montage revealed that an artist’s relationship to the surrounding reality and their view of the world, is not merely expressed in the entire process of shooting, but in their capacity to observe and present their world. In this way, themontageis linked to an artist’s world-view and one’s ideological purpose (Corrigan et al., 2010). The montage challenged linear narrative representations at the time and has henceforth been a common narrative technique adopted across films. For these reasons, the montage can be considered one of the early novelexamples of screen production enquiry.
Along with the cinematic innovation of montage, Norman McLaren’s animation experiments are screen production enquiries that advanced technology and the artistic limits of animation. McLaren pioneered a technique of creating synthetic sound in animation by etching directly on the soundtrack portion of a strip of the film in his project Scherzo (McLaren, 1939). In his experiments with painting on film strips, McLaren states that they would apply paint design straight lines, wriggly lines, dots, and other patterns directly on film strips, irrespective of the sequence of the frame. This resulted in frantic motion when projected on screen, however when supported with fast paced music, the animation was mostly synchronized. McLaren reflects that this process was the dawn of a new field to be explored. These resultant artefacts, and knowledge from McLaren’s cinematic experiments could not have been predicted, but rather emerged from the screen production process. This approach to research addresses the significance of creating visual work without knowing the outcome of the resulting artefact. McLaren’s screen production experiments have contributed towards exploring new ways of visually engaging audiences and the new knowledge that has been further adopted by several filmmakers.
To support the novelty of early cinema, contemporary studies reveal an increase in filmmakers researching their practice through various approaches. Alison Wotherspoon is a filmmaker and researcher who includes an interdisciplinary approach of participatory action research for creating her film series, Bullying in High Schools(Wotherspoon et al., 2006). Wotherspoon’s research approach allowed her to reflect on her collaborations with academia, policymakers, clients and communities while producing this screen content (Wotherspoon, 2011). Documentary filmmakers Gillian Leahy (Leahy, 1986) and Tony Dowmunt(Dowmunt, 2010)research their own documentary practices, while they position themselves as characters and narrators on their own. As filmmaking researchers, they use subjective research to gain an insider’s perspective as they are both production crew members, working behind the camera as well as appearing in front of it, as part of the film’s narrative. An additional example of screen production enquiry can be found in Bill Nichols’ modes of documentary which uses a participatory investigation as a filmmaker, narrator, and on-camera participant (Nichols, 2002). Filmmaker Sean Maher’s docu-fiction The Brisbane Line (Maher, 2011)also featured himself in the film as an on-camera participant as he explored the theoretical perspectives between urban representation in Australian film, cinematic impacts on historiography (Maher & Kerrigan, 2016). These examples validate that screen production researchers, who are filmmakers, can benefit from understanding these insider research approaches because they provide a more subjective research enquiry. This enquiry permits them to adopt a practitioner-perspective to research their own practice. For these reasons, screen production enquiry can also be considered an insider-insider perspective, where the researcher is inside the research process and a participant in the activity of study. In this study, the director of Table for Two plays the role of a screen production researcher to investigate the production decisions, designs, and iterations in shaping the artefact to experiment with new methods for storytelling and story experiences in virtual reality.
Table for Twois a PIN VR that pivots around multi-genre conversations across three tables in an urban café. It is filmed with three omnidirectional video cameras where each sequence is a one-shot production. This section discusses the director as researcher’s perspective of production challenges encountered and decisions involved in Table for Two.
Pre-production of Table for Two
The pre-production for Table for Two included the writers’ room, character development and design, rehearsals, test shoot, staging, exploration of script writing and storyboarding formats. The conceptualization of Table for Two began in the writers’ room. Here, it was discussed that a café would be a suitable space to represent multiple genres as each table could host different conversations. CVR experiences such as Pearl(Osborne, 2016), Mr. Robot VR(Esmail, 2016), New Wave(Mallal & Hjartarson, 2015), Magenta (Pylioti, 2022)suggests that the 360-degree space is an essential character to the narrative. Likewise, the café plays this role in Table for Two as it allows multiple genres and navigation teleportation across the tables. Table for Two involves genres of romance, supernatural fiction and drama selected for they diversity to express multi- genres as PIN VR. The experience was set up amidst the socio-economic events of the COVID-19 pandemic. In the plot, the pandemic affected all characters in some way and therefore becomes a part of the larger story world.There are seven main characters in the Table for Two PIN VR experience. Two characters on each of the three tables and a waiter. The waiter was utilized as a mediator between the tables. As the only character who directly interacts with all three tables, this character further encourages the navigator to teleport from one table to the next. Each character’s personality and backstory were discussed in the writers’ room. After finalizing the plot, the characteristics, and demographics were discussed. This was followed by sketches during the character design phase, which helped in recruiting the appropriate casting talent.
Several studies(Chang, 2016;Gödde et al., 2018)suggest that CVR experiences have many similarities with stage theatre such as acting for long takes, limited transitions, exaggerated acting and emphasis on body gesture. For these reasons, the casting talent of Table for Two were recruited from the local theatre group. While none of the casting talent had previously performed for PIN VR, their experience with stage theatre helped them to with perform for long takes in PIN VR. However, PIN VR encountered its own challenges with simultaneous dialogues overlapping each other, being captured from multiple 360-degree cameras and lack of live audience feedback. Therefore, the casting talent had several rehearsal sessions to synchronize their performance with other tables. Prior to the shoot, a test shoot was conducted to understand the suitable positions for the three 360-degree cameras used in the café. During the initial test shoots, the 360-degree cameras were placed in the middle of the table as the performers were conversing. This placement would result in navigation difficulties, due to which it was revised to a third member present adjacent the performers conversing. This was followed by diagrams that proposed the placement of 360-degree cameras, characters, tables, chairs, and other objects in the café. These diagrams were later revised once the shooting location was finalized.
PIN VR was a new narrative concept, due to which the cast and crew initially had difficulties in understanding their role. Various script writing methods were explored to make the script clear to the team members. A traditional, tabular, and spherical script writing format were initially explored and shared with the team members. These formats had limitations in terms of synchronization of the simultaneous parallel scripts and timeline. These limitations were addressed by a rotating cylindrical PIN VR script display (Remedios et al., 2022) as shown in figure 2. The three parallel scripts were printed and pinned up on a rotating display for the cast and crew members to make connections between the parallel scripts. A survey with the cast and crew members revealed that the display enabled them to understand their role and the interconnections in a clear manner. The script was further converted into a visual storyboard using a split screen method inspired from the frequent online meetings during the COVID-19 pandemic and split screen parallel narrative films. The split screen method allowed the cast and crew members to visually understand their gestures in the performance and synchronization with the events in the café. The elaborate pre-production process ensured all the relevant planning was in order prior to the shoot.
Production of Table for Two
The production design challenges for PIN VR included set design, cinematography, and direction. The director identified a local restaurant as the location for the PIN VR shoot. The set design team was challenged to enhance the mood of the space as per the narrative design and transform the restaurant into a café. The director discussed his vision of a Bollywood film theme for the café with the set designer. Bollywood film posters were framed as a backdrop to the conversations, however, these film posters corresponded to the genre of conversation there were representing. Along with the backdrops, the lighting and objects also reflected the genres. This resulted in the space seamlessly morphing as per the genre. To support this idea of genres, the lighting and props were also customized as per the tables. In a PIN VR experience, the 360-degree cameras see everything. Therefore, all the lighting and props were required to be integrated into the set
design. In conventional cinema, specific lighting equipment can be placed outside the view of the camera. In this experience, fairy lights, table lights, floor lamps and ceiling lights were used to light up the characters seamlessly. In addition, white table mats were used as light reflectors to the ceiling spot lights.
In conventional cinema, a cinematographer shoots the film, frames the shots, and suggests camera and lighting ideas. In CVR, the camera captures everything and in the case of Table for Two, there were three 360- degree Ricoh Theta V cameras used to capture the events of the café from different viewing perspectives. There were two cinematographers selected to take on the camera responsibilities in Table for Two. Conventional cinema offers the freedom of utilizing several camera angles and compositions such as close ups, mid shots, over the shoulder, canted angles, long shots, establishing shots, etc. However, in PIN VR, too many variations in camera positions, angles and transitions would disrupt the navigation experience. This limitation was more challenging for the cinematographer to decide the location and height suitable to view the conversations at the tables. The height was decided to be at eye level of characters seated at the café, immersing the navigator as a third guest on each table. Furthermore, the distance of each camera from the characters was important to engage the navigator with the conversation. However, the position was required to be appropriately placed from the view of the other cameras. This would ensure that there is less character movement in front or behind the camera for ease of removing the cameras and the monopods in the post- production. In conventional cinema, the cinematographer is often behind the scenes, however, the technical constraints of Table for Two production resulted in the camera operators acting as characters in the café as shown in figure 3. They were operating the 360-degree cameras through smart phones. The cinematographers clapped their hands, seconds after they started recording as a sound cue for the editing stage.
Due to previous experiences in filmmaking, the director as a researcher performed multiple roles in in Table for Two such as script writing, character design, casting, visual storyboarding, set design, art direction, dubbing, compositing, editing, ambisonics sound design and interaction design. Table for Two took four
continuous re-takes during the production phase. During the first couple of takes, the director was waiting outside the café and could not figure out if the shoot had come out well. The time taken to download each video footage and review it on the set would be too time consuming. The most organic method for the director to review the performance was to perform as a supporting actor in the café as shown in figure 3. He was pretending to read a book, however, was paying attention to acting in the café. In conventional cinema, the director at time appears in cameo roles, however, is most often behind the scenes. The situation in Table for Two required the director’s presence as an actor. The navigators were limited to teleportation points of the three genres and could not teleport to the table in which the director and the cinematographers were located. The director and cinematographers’ presence in front of the 360-degree cameras significantly contributed to the quality of acting performance by the casting talent.
Post-production of Table for Two
The post-production of Table for Two includes compositing, colour correction, dubbing, sound effects, music composition, editing, and interaction design. The compositing challenges included digitally painting over the 360-degree cameras and monopods for masking. This was necessary, so that the navigators could not see the cameras visible during the experience as the cameras visibility would make the experience appear staged.
Sound design is another significant aspect of a PIN VR post-production as it integrates dialogues, atmospheric sound effects and music to complement the 360-degree videos. In Table for Two, the wild track was captured through 360-degree cameras to avoid any other microphones visible in the video. The director and each cast member dubbed the dialogues in a sound studio. The dubbed dialogues were positioned in a 3D spherical space by exploring spatial and ambisonicspanner tool in Adobe Premier. To complement the dubbing, atmospheric sound effects were positioned in the appropriate space to simulate navigator immersion in the café. Furthermore, music composition prepares the navigator with the anticipated mood required to experience each genre. Initially, a theme was created for the experience, followed by variations of the theme as per the genre. In this way, when the navigator teleports, the music theme remains the same, reinforcing that they are in the same narrative, yet the mood changes as per the genre. In this way, the music composition played a role as a binding thread to stitch the narratives together.
The music composition process was followed by colour grading. In Table for Two, the keys aspects on the colour grading for the romance genre included brighter lights, less shadows, softer colour tonesas shown in figure 4A. The supernatural fiction genre included more contrast, intense brightness, and shadows, and darker warm colour palette to simulate ghost storytelling like experience as shown in figure 4B. The drama genre also included contrasting lighting and shadows, but not as intense as supernatural fiction as shown in
figure 4C. While seen in comparison, the differences in the colour grading for each genre are clearly visible, however, these differences appear subtle when experienced in a Head Mounted Display during navigation testing of the project.
As Table for Two proposes a PIN VR narrative experience which a navigator might not be familiar with, a short note was added before starting the experience. The note reads as follows.
Table for Two is a Parallel Interactive Narrative in Virtual Reality (PIN VR).
The navigator can teleport across three selected tables by selecting the PINs above.
This note was essential to inform the navigator that they can teleport from one table to another during the experience. However, as more of such narratives are developed in the future, the experience will not require the support of information before the runtime.Table for Two does not include any cuts or changes in camera angles, due to which the montage is created intuitively by the navigator as they juxtapose the genres. Furthermore, this study recommends reorienting each 360-video such that the table of importance is in the center. This will ensure that each genre begins with this view. Table for Two was rendered in MP4 format with specifications of a VR video with ambisonics sound. The rendering stage is where a conventional film is completed, however, PIN VR includes an additional stage of interaction.
The interaction of Table for Two was performed on 3D Vista Virtual Tour software as it permitted live teleportation. In the interaction design, hotspots were added as a locationsignage above the tables where the navigator could teleport. Once they selected that area, they shifted their orientation to that narrative at the exact point of time in the experience. Once the interlinks are completed, the PIN VR experience can be exported in web, mobile, standalone, Facebook and YouTube formats. Once rendered, the experience is preferably viewed on a Head Mounted Display; however, can also be viewed on mobile, laptop or desktop through rotation gesture, or controlled with a finger or mouse. All these steps are recommended to add value to the PIN VR experience. The following section includes a discussion on the effectiveness of screen production enquiry to gain insights from Table for Two production.
This article establishes Cinematic Virtual Reality (CVR) as one of the latest disruptions that challenged filmmakers with new approaches to design experiences for 360-degree space. This article highlights the valuable contribution of filmmakers using screen production enquiries towards their own practices with examples of the montage and the work of McLaren in early cinema, followed by examples of contemporary filmmaking research. Therefore, this method was suitable to establish new ways of storytelling in CVR. This article adopts screen production enquiry forTable for Two, a parallel interactive narrative in Virtual Reality (PIN VR).
In conventional cinema, the montagejuxtapositions are decided by the director or film editor based on their artistic choices and world views; however, in Table for Two, the story world is decided by the director and the montage is intuitively created by the navigator by juxtaposing the order of events in the café. In this way, the director and navigators are co-creators of Table for Two.Screen production reflections of McLaren highlights that the practitioner does not always know the outcome of the project, however, it is the iterative creative process that eventually shapes the outcome. Similarly, in Table for Two, iterations have taken place in test
shoots, script writing explorations, storyboarding, set design, casting talent’s improvisations and decisions which have resulted in an outcome much different from the starting point. Therefore, this study encourages screen practitioners as researchers to report, discuss and reflect upon their iterative design decisions for their creative artefacts.
The screen production investigation of Table for Two highlights the importance of space in ideating a PIN VR experience. This project suggests that a test shoot prior to the production aids the crew members to better understand how to utilize the potential of 360-degree cameras and sets. The production of Table for Two validates the effectiveness of casting talent with experience in theatre, to perform in VR experiences due to similarities of long takes, improvisations, spatial storytelling, and the use of body gestures in acting. However, synchronization of parallel storylines, performers being captured from multiple 360-degree cameras and lack of live audience feedback were more challenging in PIN VR. The rotating cylindrical PIN VR script display also enabled the cast and crew members to experience the parallel scripts at the same time, thereby enabling them to understand the concept of Table for Two and their role in a clear manner. The effectiveness of the display can be further tested in further PIN VR projects. Considering the challenges of PIN VR set design, it is recommended to integrate the lighting material and props as a part of the set. During the Table for Two shoot, the director and cinematographers were participants visible in front of 360-degree cameras for an organic synchronization of coordination, thereby drawing similarities to Leahy, Dowmunt, and Nichols’ participatory approach to screen production research. In the post-production of Table for Two, it was necessary to digitally paint over the 360-degree cameras and monopods, followed by masking performers movements to ensure that the navigator does not see any shoot equipment during the experience. The sound design process included capturing the wild track, studio dubbing, ambisonics atmospheric and dialogue positioning, and music composition with a theme that adapts from one genre to the next. The colour grading process was further developed to complement the mood for each genre. While conventional cinema completes at the rendering stage, PIN VR includes interaction design where hotspots and synchronization are performed and exported for preview in a Head Mounted Display. Despite borrowing techniques from conventional cinema for parallel narratives, long takes in stage theatre performance, and video games for interaction design; PIN VR emerges as a unique medium with alternative narrative possibilities and these considerations are recommended for designing PIN VR experiences.
Candy (2006) states that practiced-based research has given rise to new concepts and methods in the generation of original knowledge (Candy, 2006). These insights discussed in formulating Table for Two could only be gathered through a practitioner’s lens. Hence, Table for Two demonstrates how screen production enquiry can be applied for a VR narrative design to explore new ways for storytelling and story navigation. It establishesthe role of a director as a researcher’s perspective on an ever-evolving creative process. This article emphasizes the requirement for more filmmakers to investigate their creative artefacts through screen production enquiry as their unique perspectives can broaden the possibilities of creative practice.