Immersive Art & Beyond

Virtual reality is increasingly being used to capture audiences' attention and make various fields more compelling. This episode delves into a VR experience about Charles Dickens' literature and immersive public art about rising sea levels. It also features an interview with the interactive media director of the New York Times about the future of journalism.

AIRED: April 22, 2020 | 0:26:37




Roosegaarde: We live in a world where there's maybe not

a lack of technology or money

but a lack of imagination,

and I think an immersive installation

helps people to open up,

to wonder, how do you want your future world to look like?

Hammonds: That's one amazing thing that virtual reality can do,

is really foster that empathy

and make you feel in a way

that some other forms of art may not.

In VR, you can connect with someone in a different way,

and you can take some of the things

that you see in that experience home

as if they're your own experiences,

and then it transforms you in some way.

Roberts: At the core of this is the way

that you remember the experience.

You remember it as something you did rather than watched.

I think it's, like, in your mind differently,

in a maybe potentially more powerful way,

that connects you to the subject matter

in an interesting kind of fashion.



Roosegaarde: I always believe in connecting science with technology

with imagination

to wonder how we want our future world to look like.

I'm from the Netherlands, so we live below sea level,

so without technology, without design,

we would literally drown a horrible death.

So every tree that you see,

where I built my tree hub, is planted.

It's man-made.

So we're trying to find a harmony

between nature and technology.

We live with water. We fight with water every day

for more than 1,000 years,

and WATERLICHT, in that way, is a very special project

because it literally is sort of virtual floods

showing how high water level would be in the nearby future,

and it's sort of scary but also beautiful,

and that's why we have thousands of people coming,

wondering, enjoying,

thinking how they want their future world to look like.

It's a collective experience.

It's something very tactile.

You've got to go there.

It's sometimes freezing, sometimes it rains.

That's all part of it.

And, yeah, I think for me, art is an activator

to make people more curious about the future.



[ Cheers and applause ]

But you'll see WATERLICHT at location.

It's in the middle of New York,

so there's a lot of light pollution

from the surrounding buildings.

So all the buildings, individual,

all the windows were shut down

or being covered with black cloth to reduce light.

All of these people got a letter.

Everybody had to agree.

So we first had to make this whole area

in the middle of New York dark

so that the artwork light would become visible.

So we needed darkness in order to light to become seen.

It just is sort of a nice metaphor.


We all know we need to change,

but somehow we'd rather hold on to what we know

than we invest in something new

because we're sort of scared, so...

And I think it's sort of biological

even that the part in our brain

which we base our decision is not influenced by numbers,

but more by emotions and experiences,

so maybe there is something to be learned

from these immersive experience,

that it opens up a dimension within ourselves

to be more open, to be more curious,

and maybe that's one of the crucial qualities

that we're going to need, as humans,

to survive in the upcoming 10, 15 years.

All the astronauts, we have two dozen astronauts

going up in space, all came back,

and they saw the blue marble, planet Earth, from a distance,

and they all said, "Oh, we're doing it completely wrong.

We should become more sustainable,

more engaging, better for the planet,"

so the fact that they experienced the Earth

from a different perspective

changed their mind,

changed their way of being, you know,

and so it's a very concrete example

of how immersive experience can change your perception

and therefore, yeah, your way of living,

and I feel part of that kind of thinking.


So PRESENCE is really first time

we do indoor big solo exhibition in a museum.

The museum really wanted to have a new type of statement,

so the first thing we did

is remove these horrible "Please do not touch" signs

and say, "Everything is please touch."

So we will design it in such a way

that it can be touched, you know, people can --

You can clean it. It's okay. It's made for it.

And secondly is I wanted that your presence,

your physical being in that space,

would be the main activator for the work.

So you create the artwork, and the artwork creates you.

Without you, it does not exist.

And that's what happens.

It literally shows the imprint of you being there,

and it starts more like you're in a copy machine.

Like, it sort of scans you.

It reveals imprints.

It's very intimate.

It's very personal.

And I think it's really because people want

this physical interaction.

They see it online,

but online activates offline,

and they want to go there. They want to hang out.

They want to share it. They want to experience it.

And that's real, you know?

So it's a platform,

and, in the end, the social interaction,

and the interaction that people generate

with themselves or each other,

that's the real artwork.


We live in a world where humans are sort of

being reduced to robot food.

We're sort of feeding our computer screens

with our hopes, our dreams, our desires,

and what do we get back? A like.

You know, that's a bad deal.

So what happens when technology jumps out of the screen,

and what happens when technology starts helping us

instead of we are just feeding it?

And that's a shift that you're seeing

from people putting their phone down

or desire for more biological food.

You know, like, you can see it sort of shifting

from euphoria 10 years ago to a more, yeah, neutral.

So I think we will live in a post-screen era

where the screen is still there,

but we just don't look at it that much anymore,

and it becomes embedded.

And I think that's a great new way of interfacing.

For example, our Van Gogh bicycle path in the area

where Van Gogh literally lived and worked,

charged at daytime via the sun, glows at night,

so it's about history, as an homage,

as a celebration of the famous Dutch painter,

but also a statement about future.

How can we make energy-neutral landscapes?

Why do we have streetlights burning

the whole night normally?

Can we not do that with sun?

So it's a new type of interfacing

and a new type of connecting.


There is definitely a hunger for it,

especially the younger generation.

We can sense that.

They are really open for it, and, you know,

it starts maybe a bit naive,

like to collect likes for Insta,

but that's just the beginning of building

up a new language, you know? That's just, you know,

that's how you learn to speak in a new way.

First, you learn A, B, C.

And then you make a sentence, and then you write a book,

and then you write a great book, you know?

So I think society is sort of getting used to a new grammar.

What is interesting is, I think,

we moved completely away from the LEDs,

the cables, the wires, the beamer projection bullshit

and really tried to bring it into tactility

and really use, for example, the sun to charge,

and it glows, or headlights of cars,

which where the light is reflected

to make energy neutral buildings,

so it becomes more nature.

We learn from nature,

and that allows the artworks to work

and remain working for 1,500 years

in a harsh environment like public space.


A good immersive experience creates a memory,

and it's something that you remember.

So, like, what do you really remember?

Not so much,

and that's what I see

when I meet with the people.

I see it in their eyes, and it's the notion of wonder,

and that's opening up.

And then you get science, money, technology to make it happen.

But it's the beginning of change, yeah,

or at least that's the reason why I'm doing it.


Hammonds: We're here at the Virtual Arcade

for the Tribeca Immersive Exhibition.

It really has developed into more of a storytelling festival,

so, obviously, have film as our backbone,

and that's in the DNA of the festival,

but we embrace stories on all screens.

We just recognized that people are using

new technologies to tell stories,

and we want to be sure to encourage that

and just give them a great platform to share it.

And just in the past year,

360 cameras have dropped dramatically in price.

This was our first year opening up for submissions,

so we had cold submissions.

We had about 250 submissions

versus, you know, in the past,

myself and my co-curator have been traveling the world

to find good new work to showcase.

And to be able to open up submissions

and see people that are tinkering in their basements

and their bedroom, and not just with 360 cameras,

also with fully realized environments

in, you know, a game engine,

it democratizes the art form,

so it's exciting times, yeah.


We have pieces like "Draw Me Close,"

which kind of melds the idea of immersive theater and VR,

where you have a live motion-captured actress

who you can actually interact with in a virtual environment.

These are amazing leaps

in terms of just how to tell stories

and new additions to, you know, to making great work.

It's especially powerful for documentary storytelling.

It's also a call to action.

To be able to hold that kind of eye contact

with someone that you might not have a chance to be around

or have thought of being around

or may have a fear of, for that matter --

I think that that's really powerful

to break through your preconceived notions.

A piece like "Becoming Homeless,"

they do great work in actually using virtual reality

for their experiments.

They're recording people's responses here

and what it is to become homeless,

to be threatened with eviction,

to interact with other people

that are, you know, enduring this epidemic.

So it's a powerful thing.

I mean, I think even in narrative,

your approach as a filmmaker,

as a screenwriter, as a playwright,

is always you want your audience to feel connected to the piece.

You want them to feel connected to the characters.

I think that that's possible in a way

that it has not been before through virtual reality,

so that's why we see a lot of people

from different mediums trying it out.



"Vestige" is based on interviews

I've been doing with a woman called Lisa from Utah.

Her husband died 2 years ago.

I started talking to her six months after his death,

and this kind of follows the journey pathway

of her memories over time,

so kind of looking at how we remember someone

during the process of grief

and how that changes over the time of grief, as well.


This has kind of existed as an audio piece for a long time,

and building the narrative and building the sound design

before actually kind of curating the physical experience.

They kind of cross over at some point,

but I know the power of this piece

is when you are in that space and moving around

and Lisa is in the space with you.

That's where it really kind of becomes its own experience.


People still enjoy reading books and, you know,

feel, in some ways, more immersed in those worlds

than in the most technologically advanced immersive medium,

but there's something for me as an artist

that is fascinated by what you can do

and what you can achieve in that space,

and I think, for "Vestige," what I'm trying to explore

is the intimacy of that situation.


When people feel close to another person,

when someone is in your space and talking to you

and talk to you about something very deeply emotional,

you connect with them on a whole different level.

That, for me, is something I've not been able to experience

in any other medium before,

so I think that's the true power that VR can give to a viewer,

being able to experience things

from other perspectives in some way,

and being able to apply your own thoughts and feelings

to, you know, the situation.


I think the story is still the most important part of that,

but it projects it into a different realm,

so it depends what you want to achieve with the piece,

depends on whether it suits being a flat-screen film

or whether it should be a sculpture or a painting.

It could be, you know, all different kind of mediums,

but if you want to achieve something

which is completely intimate between one person and another,

that is where you can make something

which is beyond other mediums within VR.


One thing I'm really interested in is emergent narratives,

so narratives that you -- It's not a linear path.

You explore and find the narrative within a piece.

Within this, there's moments which change the story

depending on where you are and what you're looking at,

and so that becomes a new tool in storytelling.

You know, there's --

I guess you can kind of hark back

to choose-your-own- adventure-style books,

but when you can do it without that cognition of,

like, I make the decision to change the page,

instead we just see the variables

of where someone is and what they're looking at a time,

and then switch the story's dependent.

So all that kind of stuff

is going to completely change how we tell stories

and how we think about writing stories, as well.



Roberts: Virtual reality is, I guess, in the general sense,

the idea that we can put you somewhere where you are not.


So we have a longer history with the things

that fall into the virtual reality umbrella.

So that began --

At the end of 2015, we created an app called NYTVR,

which would host these virtual reality experiences,

and we paired with Google,

who at the time had just come out with this product

called Google Cardboard,

which is this kind of amazing thing because it said, you know,

"Here's this new medium that no one really has access to,

or so they think.

In fact, everyone is already walking around

with little virtual reality machines in their pockets --

their cell phones."

And with nothing more than a kind of

folded piece of cardboard

and a couple plastic lenses,

you could unlock that capability

and give everyone virtual reality.

Using our antiquated print delivery mechanism

of legacy media,

we were able to bring virtual reality

to millions of people by literally mailing

a million Google Cardboards

with people's Sunday's paper.

[ Foreign chatter ]

Our first virtual reality piece,

which was called "The Displaced,"

which was a story about the refugee crisis,

and it was important, I think, to --

While we were bringing this idea to many people, to show --

And, you know, it's also something

that can be used for serious journalism.

It can be used to tell the important stories of the day,

and it doesn't just have to be kind of a fun,

gimmicky kind of thing.


This is why virtual reality does better

than all the other mediums --

creating a sense of presence,

giving a sense of being there

gives you a different connection to the material, I think,

than other mediums have,

where there is a bit of a separation created by that,

you know, box, the frame.

So the frame is gone,

and I should clarify that we're not thinking,

"Oh, here comes virtual reality, so we don't write stories now,"

or something or, "We don't take photographs."

It's additive.

It's a new angle that we can give

a new kind of perspective on the story.

We created a virtual reality trip to Pluto

that we did in collaboration with NASA.

So this was just the most amazing material.

They, you know, I think in 2004, somewhere around that,

they launched the New Horizon spacecraft

which then traveled for three billion miles,

10 years, finally just got within 1,000 miles of Pluto

and took photographs, and we used that, those photos

and that data that NASA collected,

to give you a story about the experience

where you're kind of out there,

waiting for New Horizons to arrive,

and then you travel with New Horizons to the planet,

and then we actually put you down,

standing on the planet itself, based on accurate extrapolations

that we built together with NASA.

I mean, it's just such an engaging experience,

such a better way to engage an audience,

I think, in this material.

I think there's an interesting level,

maybe you could even say, with transparency

since this is a journalistic context.

Even with the best intentions, when you take a photograph,

or you're filming something,

you're making a crop.

So you're making all of these, you know,

at-the-moment editing decisions

of what you're including and excluding from the frame,

and you could argue that giving someone,

you know, a virtual reality view

or a 360 view or an omnidirectional view,

any way you want to kind of describe it,

it gets rid of the crop.

So you're giving to your viewer, "Here is the whole scene,

unedited in a sense,"

from, of course, from a very particular perspective.

Can't get around that, but it's still giving --

It's giving a little bit more, I think, to the viewer.


Another example was an attempt to

kind of use the power of the Times' archives,

and this was around the piece that we did

for the Rio Olympics, actually.

It was called "The Modern Games,"

and the concept was, "What if we could use...

Alright. What can we do with a virtual reality in the Olympics?

Well, what if we could recreate

the most inspiring moments

in the Olympic games throughout history,

since the beginning of the modern Olympics in 1896,

and let you stand there?"


It was based on all these incredible photos

that were in our archives

of, like, you know, Babe Didrikson in 1932

throwing a javelin,

and we turned these archival photos

into environments that you could stand in.

So when Babe is holding that javelin,

you're standing in the Los Angeles Colosseum

or whatever it was called in 1932, in front of her,

and now that photo is in full stereoscopic 3-D

as she's holding that javelin,

but it was all done with projection mapping

kind of techniques with the photography,

so you're really still looking at the real photography

even when you're in that.


You know, one of the things I've liked about it

is that you're really in a single thing

without distraction.

It kind of swims up against the rest of media in a way,

where, like, I'm watching TV,

but I'm running my laptop on the Internet.

I'm also checking Twitter, and I'm also doing this.

And in virtual reality, you're like,

"No. You're going to do just this for eight minutes,

and you're going to be completely immersed

in this story,

and it's going to take over everything."

I think people also kind of crave that a little bit.

It's such a fragmented, crazy world now

where just this just information overload

that it's nice to just be concentrating on a single thing

for a little bit and being really,

you know, inside of it, having it take over everything.

You know, the written word is not going away, but it's --

There are so many more opportunities now

to convey information,

and the written word is not always

the most efficient or most effective way to do that.

You wouldn't write thousands of words

so that somebody knows exactly what a painting would look like.

You show them a photo of it.

You don't have to always tell.

You can just show,

and I think virtual reality is part of that.

It is sometimes the shortest line

between what we're trying to communicate

and understanding for people,

and, you know, it's still an open question

as to the best ways that that actually happens

because it's such a new medium,

but, you know, we're trying it out, discovering it.


Hammonds: It's a very exciting time

in the development of virtual reality.

Creators and developers are working hand-in-hand

to really push those boundaries and create great experience

that are totally immersive and totally transport you.

I really feel like the sky is the limit.

Bradbury: Every single different media that emerges,

it finds a new pathway of telling stories,

and, in VR, I think it's going to be the same thing.

I think, for the first time, you can actually take variables

from the person that's viewing the experience

and put them into the experience.

That's what I think is the power of it all.

The point is to transport you to a place,

to make you feel like you're there.

Then you could argue that the more agency

that you give to somebody once they're there,

the more that they're going to feel the power of that presence,

of being there.

Roosegaarde: To drag it into an experience beyond the numbers

or the fact sheets

is a way to trigger a curiosity

and to engage people in a more emotional or interactive way.

It's also, when I look at myself,

I change because of experiences, not just because of numbers.

So if we want to create a world

which we deal with climate change

or rising sea level,

these kind of experience might be a key factor

to create change.

And it's great to be part of that

and fuel that with new stories, yeah.







  • ios
  • apple_tv
  • android
  • roku
  • firetv