The science of creativity
New immersive ways of creating and consuming content across virtual, augmented and mixed reality mean content creators and technologists need to work together more closely than ever before.
Professor David Bull, Director of the University of Bristol's Vision Institute, spoke at the Beyond creative conference about the challenges of bringing creators and technology experts together to create new technologies and experiences.
The following has been taken from Professor Bull’s talk.
Technology to drive creativity
Arguing that it is technology that drives the creative sector, rather than the reverse, Professor Bull said:
“In many ways, technology leads the creative sector. It’s an enabler, analyser and inspiration for creative content. And the human hunger for interesting, engaging and informative content is driving technology.
“Technology has had an impact on creative pieces, from the oldest drawing instrument 100,000 years ago to the printing press, film, global communications, smart devices, and immersive technology.”
Bull offers the Royal Shakespeare Company’s (RSC) 2016 production of Shakespeare’s Tempest as an example of this relationship. The RSC worked with Intel and performance capture specialists, Imaginarium Studios, to use capture technology to translate an actor’s facial expressions into an animated version of the character Ariel that the other actors interacted with in real-time.
“It’s all about the people and understanding audiences. 50% of neurons in the human brain are associated with processing visual information, compared to 10% for hearing and 10% for other senses. This explains why we are so stimulated by vision and why it can engage, entertain and inform us.”
While technology and art have been successfully integrated before, Bull believes that it is important to proceed with caution.
“There’s so little in the creative world that isn’t touched by technology. But you have to be careful – in the 2008/09 research world, you couldn’t get funding without doing 3D, thanks to Avatar and other successful 3D movies. Yet, as of 2018, there were no TV manufacturers making 3D displays.”
“The current wave of enthusiasm for virtual reality (VR), augmented reality and mixed reality is creating the same level of hype [as 3D technology]. That’s great, it gets interest in the technology and allows it to be developed. It creates the investment for that technology.
Overcoming technological challenges to reach market potential
Bull said that immersive technology needs to overcome a number of challenges to reach its predicted market size of $200 billion (around £155 billion) by 2022.
One of the largest, he says, is the ‘vergence-accommodation conflict’, known as the VAC. Vergence is the way in which our eyes look at targets that are different distances away. Accommodation is how we focus on these targets.
The conflict occurs when our eyes look at 3D images, which are created by showing each eye a slightly offset image. The eyes have to try and accommodate, or focus, on that image, while simultaneously converging to a far-off distance. This means they have to ‘uncouple’ the two processes, which can lead to fatigue and discomfort.
Working together to overcome immersive challenges
Bull said that other problems, including sensory issues and data restrictions, need to be addressed before immersive content is more widely adopted.
“In an enclosed VR environment you sacrifice your main senses of hearing and vision. The impact on the other senses becomes far more accentuated. The misalignment of the senses, especially the vestibular system [responsible for your sense of balance] can be a big issue.
“We also need to deliver the content. 5G has potential, but there are still many challenges we need to address. The bit rates we demand for immersive experiences are huge. We need compression ratios of thousands to one – that’s hugely challenging. At the same time, you have to preserve those immersive properties of what was created in the studio, when delivered to the user.”
Measuring the impact of immersive technology
Finding a way to measure the impact of immersive content before widespread adoption is essential. The challenge, Bull argues, is in separating the content from the technology to determine impact.
“How do you know whether the impact is from the narrative or the technology? This is something we’ve been working on at the University of Bristol with the BBC’s Blue Planet. We realised that we define our engagement as attention – a measure of that attention is how long it takes your brain to switch out of one task and into another. To measure the [impact of the] technology, we need to subtract the narrative out of that equation.”
The creative continuum
Bull’s conclusion is that immersive experiences demand an integrated approach from both technology and creative sectors.
“Of course, technology underpins everything. Where would we be now if scientific developments weren’t informing creative processes? But we also need to recognise, understand and exploit those interdependencies across the design, capture, delivery and consumption processes, because they all contribute to the experience.
“To do that, we need people working together. We need psychologists, we need engineers, we need computer scientists, and we need people in the creative arts.”
Working together to captivate the audience of the future
The Industrial Strategy Challenge Fund’s Audience of the Future challenge is supporting businesses, researchers and technology experts to work with the UK’s most talented storytellers. Together, they are creating exciting, immersive experiences to captivate the public.
The challenge exists to ensure that the UK remains at the forefront of the creative revolution.