Expert opinion: Live theatre and cinema meet augmented reality

Roma Patel, a Nottingham Trent University researcher developing the storyline and interaction for the augmented reality app, Riot 1831@ Nottingham Castle, discusses the importance of storytelling in museums.

Camera icon
Digital model of Nottingham Castle created using photogrammetry

As the national Museums at Night celebration gets under way, Nottingham Castle is launching a cutting-edge augmented reality app to bring exhibits to life. Roma Patel, a Nottingham Trent University researcher, has developed the storyline and interaction for this new project, Riot 1831@ Nottingham Castle. She discusses the importance of storytelling in museums.

Museum objects are not only evidence of our past, but have stories to tell. Telling stories has preserved the identities of individuals and communities and was one the earliest ways of managing knowledge. There was no real distinction between history and story.

Storytelling that is interwoven into the museum visitor's experience can encourage a more subjective and emotional connection to the past by layering competing accounts of historical events beyond the single voice of 'authority'. The flexibility of mobile technologies gives museums the possibility to tell stories in different ways, to share and create a living history experience for their visitors. Digital storytelling can bridge the gap between explicit information and implicit knowledge that museums and heritage sites often struggle with balancing.

Riot 1831@ Nottingham Castle is a project supported by the Digital R&D Fund for the Arts – Nesta, Arts and Humanities Research Council and public funding by the National Lottery through Arts Council England.

Nottingham Castle was established as the first municipal museum and art gallery outside London in 1878. A castle site since the 11th Century, it has a chequered history of murders, sieges, and intrigue. The medieval castle was destroyed during the Civil War (1642 – 1651) and replaced by a Ducal Palace in 1674. On an October night in 1831, the palace was set alight by rioters protesting against the reform bill. That night left a legacy, a unique combination of diverse first-hand witness accounts and museum objects.

The exhibition explores the theme of Riots focusing on the 1831 Nottingham Reform Bill Riot and its effects on the Castle museum's site through a digitally integrated augmented reality (AR) experience. It incorporates AR as an active storytelling medium to develop an understanding of the relationships between the 'real' and 'virtual' objects.

The curatorial relationship between the digital and physical exhibition are carefully considered, the AR app was developed in parallel with the re-design of a new exhibition to bring a new perspective to the historical collection. To create an intermingling of the virtual and the real objects, the past and the present, iPads are dispersed throughout the space, positioned in front of paintings and objects in display cabinets making the gallery accessible to all visitors.

Although there are examples of AR in museums, there are very few that exploit the benefits of the dynamic elements of three dimensional virtual environments (computer games) and animation. AR is an evocative storytelling medium when it draws upon and synthesises theatrical site specificity, the real time capabilities of computer games and cinematic aesthetics with traditional museum practices, particularly interpretive narrative.

Although the story follows a linear narrative that begins with the news of the Reform Bill's rejection in parliament and ends with the castle on fire, the visitor's experience is non-linear. Their experience may not follow a chronological journey since the exhibition space has three different entrances. The visitor's curiosity will have to lead their exploration and interpretation. Each AR story episode was designed to be viewed independently.

The context in which the experience is viewed is also significant; a visitor using their smart phone will have a different experience to the visitor using the iPad provided in the exhibition.

Camera icon
James Marriot, gate keeper at the time of the riot, recreated digitally

The design and construction of the AR stories used storytelling techniques from cinema, theatre and computer games and also had an understanding of the limitation and boundaries of the digital medium. In the case of a wall mounted painting of the Castle Gatehouse – the augmentation 'brings the painting to life' similar to a Trompe l'oeil effect, but also beyond the frame, it extends the virtual performance into the Gallery space. During the course of the story the visitor moves around their mobile device to view the virtual scene. Each interaction with the silhouetted Rioters advances the story by moving the rioters closer to the Gate.

The spatial relationship is a unique element of AR and can be used to great effect to immerse the viewer. The 360 degree panoramic real-time augmentation in the real space means the visitor is surrounded by an unbroken view of the virtual images within their surroundings. The viewer's field of vision is restricted by the dimensions of the screen and the extent of the virtual projections into the space can only be seen within this frame (screen). The visitor controls their view by moving their device around the space as though viewing a panoramic shot in a film. For example the virtual image of a 1831 town map made from lace threads hovers over a real scale model of the town.

The intention of this project is not to replicate photorealism realism, nor is it a virtual re-enactment of the 1831 Riots. Instead, it aims to create a dramatisation of the eyewitness accounts of the events, designed to help the visitor feel connected to the people of the past. The experience of a virtual environment lets the museum visitor have free movement of the image within his vantage point. The live and virtual cameras coexist and the compositional spatial elements are arranged and rearranged onto the museum object and space.

The digital layers can become a "Spatial Montage" as described in Lev Manovich's essay 'What is Digital Cinema? of simultaneous images 'coexisting' on a computer screen. However, AR takes this idea a step further as the events and images progress through coexisting physical and virtual narrative paths and it has, as though the gallery space, become a promenade performance where the AR stories, the museum objects and the site merges.

Roma Patel

Researcher for Nottingham Trent University, School of Art & Design

  • Notes for editors

    Press enquiries please contact Kirsty Green, Press and Public Affairs Manager, on telephone +44 (0)115 848 8799, or via email.

    To deliver the best possible exhibition experience, Nottingham Castle is working collaboratively with expert partners from both the world of academia and cutting edge visual communication:

    The £125,000 project is supported by the Digital R&D Fund for the Arts.

    The Digital R&D fund for the Arts is a £7 million fund to support collaboration between organisations with arts projects, technology providers, and researchers. It is a partnership between Arts Council England, the Arts and Humanities Research Council and Nesta. It wants to see projects that use digital technology to enhance audience reach and / or develop new business models for the arts sector. With a dedicated researcher or research team as part of the three-way collaboration, learning from the project can be captured and disseminated to the wider arts sector. Every project needs to identify a particular question or problem that can be tested. Importantly this question needs to generate knowledge for other arts organisations that they can apply to their own digital strategies.

    Nottingham Trent University, School of Art & Design researchers Roma Patel and Deborah Tuck are working closely with Adrian Davies of Nottingham Castle to develop storyline, script and interaction. Roma Patel is also co-ordinating the project academic research exploring the technology as interpretation.

    Nottingham based Hot Knife Digital Media, specialists in animation and digital content, are the technology providers creating the 3D Augmented Reality and App design including modelling, animation and interaction.

    Dr Richard Gaunt, University of Nottingham, School of History, will play an essential role in the historic re-interpretation of the riots and the 4th Duke of Newcastle, who owned Nottingham Castle at the time. Dr Stuart Reeves, University of Nottingham, School of Computer Science, will provide expertise in recording how visitors actually use the new technology. Dr Trevor Foulds is bringing specialist knowledge of the Ducal Palace. Many staff and volunteers at Nottingham Castle are uncovering exciting new connections through research that are helping to develop rich content for the new exhibition, incorporating the digital project.

    The new exhibition will initially open on May 16, for visitor testing, then formally on 4 July .

Expert opinion: Live theatre and cinema meet augmented reality

Published on 16 May 2014
  • Category: Research

Still need help?

+44 (0)115 941 8418