Immersive Art & Beyond
Virtual reality is increasingly being used to capture audiences' attention and make various fields more compelling. This episode delves into a VR experience about Charles Dickens' literature and immersive public art about rising sea levels. It also features an interview with the interactive media director of the New York Times about the future of journalism.
Roosegaarde: We live in a world where there's maybe not
a lack of technology or money
but a lack of imagination,
and I think an immersive installation
helps people to open up,
to wonder, how do you want your future world to look like?
Hammonds: That's one amazing thing that virtual reality can do,
is really foster that empathy
and make you feel in a way
that some other forms of art may not.
In VR, you can connect with someone in a different way,
and you can take some of the things
that you see in that experience home
as if they're your own experiences,
and then it transforms you in some way.
Roberts: At the core of this is the way
that you remember the experience.
You remember it as something you did rather than watched.
I think it's, like, in your mind differently,
in a maybe potentially more powerful way,
that connects you to the subject matter
in an interesting kind of fashion.
Roosegaarde: I always believe in connecting science with technology
to wonder how we want our future world to look like.
I'm from the Netherlands, so we live below sea level,
so without technology, without design,
we would literally drown a horrible death.
So every tree that you see,
where I built my tree hub, is planted.
So we're trying to find a harmony
between nature and technology.
We live with water. We fight with water every day
for more than 1,000 years,
and WATERLICHT, in that way, is a very special project
because it literally is sort of virtual floods
showing how high water level would be in the nearby future,
and it's sort of scary but also beautiful,
and that's why we have thousands of people coming,
thinking how they want their future world to look like.
It's a collective experience.
It's something very tactile.
You've got to go there.
It's sometimes freezing, sometimes it rains.
That's all part of it.
And, yeah, I think for me, art is an activator
to make people more curious about the future.
[ Cheers and applause ]
But you'll see WATERLICHT at location.
It's in the middle of New York,
so there's a lot of light pollution
from the surrounding buildings.
So all the buildings, individual,
all the windows were shut down
or being covered with black cloth to reduce light.
All of these people got a letter.
Everybody had to agree.
So we first had to make this whole area
in the middle of New York dark
so that the artwork light would become visible.
So we needed darkness in order to light to become seen.
It just is sort of a nice metaphor.
We all know we need to change,
but somehow we'd rather hold on to what we know
than we invest in something new
because we're sort of scared, so...
And I think it's sort of biological
even that the part in our brain
which we base our decision is not influenced by numbers,
but more by emotions and experiences,
so maybe there is something to be learned
from these immersive experience,
that it opens up a dimension within ourselves
to be more open, to be more curious,
and maybe that's one of the crucial qualities
that we're going to need, as humans,
to survive in the upcoming 10, 15 years.
All the astronauts, we have two dozen astronauts
going up in space, all came back,
and they saw the blue marble, planet Earth, from a distance,
and they all said, "Oh, we're doing it completely wrong.
We should become more sustainable,
more engaging, better for the planet,"
so the fact that they experienced the Earth
from a different perspective
changed their mind,
changed their way of being, you know,
and so it's a very concrete example
of how immersive experience can change your perception
and therefore, yeah, your way of living,
and I feel part of that kind of thinking.
So PRESENCE is really first time
we do indoor big solo exhibition in a museum.
The museum really wanted to have a new type of statement,
so the first thing we did
is remove these horrible "Please do not touch" signs
and say, "Everything is please touch."
So we will design it in such a way
that it can be touched, you know, people can --
You can clean it. It's okay. It's made for it.
And secondly is I wanted that your presence,
your physical being in that space,
would be the main activator for the work.
So you create the artwork, and the artwork creates you.
Without you, it does not exist.
And that's what happens.
It literally shows the imprint of you being there,
and it starts more like you're in a copy machine.
Like, it sort of scans you.
It reveals imprints.
It's very intimate.
It's very personal.
And I think it's really because people want
this physical interaction.
They see it online,
but online activates offline,
and they want to go there. They want to hang out.
They want to share it. They want to experience it.
And that's real, you know?
So it's a platform,
and, in the end, the social interaction,
and the interaction that people generate
with themselves or each other,
that's the real artwork.
We live in a world where humans are sort of
being reduced to robot food.
We're sort of feeding our computer screens
with our hopes, our dreams, our desires,
and what do we get back? A like.
You know, that's a bad deal.
So what happens when technology jumps out of the screen,
and what happens when technology starts helping us
instead of we are just feeding it?
And that's a shift that you're seeing
from people putting their phone down
or desire for more biological food.
You know, like, you can see it sort of shifting
from euphoria 10 years ago to a more, yeah, neutral.
So I think we will live in a post-screen era
where the screen is still there,
but we just don't look at it that much anymore,
and it becomes embedded.
And I think that's a great new way of interfacing.
For example, our Van Gogh bicycle path in the area
where Van Gogh literally lived and worked,
charged at daytime via the sun, glows at night,
so it's about history, as an homage,
as a celebration of the famous Dutch painter,
but also a statement about future.
How can we make energy-neutral landscapes?
Why do we have streetlights burning
the whole night normally?
Can we not do that with sun?
So it's a new type of interfacing
and a new type of connecting.
There is definitely a hunger for it,
especially the younger generation.
We can sense that.
They are really open for it, and, you know,
it starts maybe a bit naive,
like to collect likes for Insta,
but that's just the beginning of building
up a new language, you know? That's just, you know,
that's how you learn to speak in a new way.
First, you learn A, B, C.
And then you make a sentence, and then you write a book,
and then you write a great book, you know?
So I think society is sort of getting used to a new grammar.
What is interesting is, I think,
we moved completely away from the LEDs,
the cables, the wires, the beamer projection bullshit
and really tried to bring it into tactility
and really use, for example, the sun to charge,
and it glows, or headlights of cars,
which where the light is reflected
to make energy neutral buildings,
so it becomes more nature.
We learn from nature,
and that allows the artworks to work
and remain working for 1,500 years
in a harsh environment like public space.
A good immersive experience creates a memory,
and it's something that you remember.
So, like, what do you really remember?
Not so much,
and that's what I see
when I meet with the people.
I see it in their eyes, and it's the notion of wonder,
and that's opening up.
And then you get science, money, technology to make it happen.
But it's the beginning of change, yeah,
or at least that's the reason why I'm doing it.
Hammonds: We're here at the Virtual Arcade
for the Tribeca Immersive Exhibition.
It really has developed into more of a storytelling festival,
so, obviously, have film as our backbone,
and that's in the DNA of the festival,
but we embrace stories on all screens.
We just recognized that people are using
new technologies to tell stories,
and we want to be sure to encourage that
and just give them a great platform to share it.
And just in the past year,
360 cameras have dropped dramatically in price.
This was our first year opening up for submissions,
so we had cold submissions.
We had about 250 submissions
versus, you know, in the past,
myself and my co-curator have been traveling the world
to find good new work to showcase.
And to be able to open up submissions
and see people that are tinkering in their basements
and their bedroom, and not just with 360 cameras,
also with fully realized environments
in, you know, a game engine,
it democratizes the art form,
so it's exciting times, yeah.
We have pieces like "Draw Me Close,"
which kind of melds the idea of immersive theater and VR,
where you have a live motion-captured actress
who you can actually interact with in a virtual environment.
These are amazing leaps
in terms of just how to tell stories
and new additions to, you know, to making great work.
It's especially powerful for documentary storytelling.
It's also a call to action.
To be able to hold that kind of eye contact
with someone that you might not have a chance to be around
or have thought of being around
or may have a fear of, for that matter --
I think that that's really powerful
to break through your preconceived notions.
A piece like "Becoming Homeless,"
they do great work in actually using virtual reality
for their experiments.
They're recording people's responses here
and what it is to become homeless,
to be threatened with eviction,
to interact with other people
that are, you know, enduring this epidemic.
So it's a powerful thing.
I mean, I think even in narrative,
your approach as a filmmaker,
as a screenwriter, as a playwright,
is always you want your audience to feel connected to the piece.
You want them to feel connected to the characters.
I think that that's possible in a way
that it has not been before through virtual reality,
so that's why we see a lot of people
from different mediums trying it out.
"Vestige" is based on interviews
I've been doing with a woman called Lisa from Utah.
Her husband died 2 years ago.
I started talking to her six months after his death,
and this kind of follows the journey pathway
of her memories over time,
so kind of looking at how we remember someone
during the process of grief
and how that changes over the time of grief, as well.
This has kind of existed as an audio piece for a long time,
and building the narrative and building the sound design
before actually kind of curating the physical experience.
They kind of cross over at some point,
but I know the power of this piece
is when you are in that space and moving around
and Lisa is in the space with you.
That's where it really kind of becomes its own experience.
People still enjoy reading books and, you know,
feel, in some ways, more immersed in those worlds
than in the most technologically advanced immersive medium,
but there's something for me as an artist
that is fascinated by what you can do
and what you can achieve in that space,
and I think, for "Vestige," what I'm trying to explore
is the intimacy of that situation.
When people feel close to another person,
when someone is in your space and talking to you
and talk to you about something very deeply emotional,
you connect with them on a whole different level.
That, for me, is something I've not been able to experience
in any other medium before,
so I think that's the true power that VR can give to a viewer,
being able to experience things
from other perspectives in some way,
and being able to apply your own thoughts and feelings
to, you know, the situation.
I think the story is still the most important part of that,
but it projects it into a different realm,
so it depends what you want to achieve with the piece,
depends on whether it suits being a flat-screen film
or whether it should be a sculpture or a painting.
It could be, you know, all different kind of mediums,
but if you want to achieve something
which is completely intimate between one person and another,
that is where you can make something
which is beyond other mediums within VR.
One thing I'm really interested in is emergent narratives,
so narratives that you -- It's not a linear path.
You explore and find the narrative within a piece.
Within this, there's moments which change the story
depending on where you are and what you're looking at,
and so that becomes a new tool in storytelling.
You know, there's --
I guess you can kind of hark back
to choose-your-own- adventure-style books,
but when you can do it without that cognition of,
like, I make the decision to change the page,
instead we just see the variables
of where someone is and what they're looking at a time,
and then switch the story's dependent.
So all that kind of stuff
is going to completely change how we tell stories
and how we think about writing stories, as well.
Roberts: Virtual reality is, I guess, in the general sense,
the idea that we can put you somewhere where you are not.
So we have a longer history with the things
that fall into the virtual reality umbrella.
So that began --
At the end of 2015, we created an app called NYTVR,
which would host these virtual reality experiences,
and we paired with Google,
who at the time had just come out with this product
called Google Cardboard,
which is this kind of amazing thing because it said, you know,
"Here's this new medium that no one really has access to,
or so they think.
In fact, everyone is already walking around
with little virtual reality machines in their pockets --
their cell phones."
And with nothing more than a kind of
folded piece of cardboard
and a couple plastic lenses,
you could unlock that capability
and give everyone virtual reality.
Using our antiquated print delivery mechanism
of legacy media,
we were able to bring virtual reality
to millions of people by literally mailing
a million Google Cardboards
with people's Sunday's paper.
[ Foreign chatter ]
Our first virtual reality piece,
which was called "The Displaced,"
which was a story about the refugee crisis,
and it was important, I think, to --
While we were bringing this idea to many people, to show --
And, you know, it's also something
that can be used for serious journalism.
It can be used to tell the important stories of the day,
and it doesn't just have to be kind of a fun,
gimmicky kind of thing.
This is why virtual reality does better
than all the other mediums --
creating a sense of presence,
giving a sense of being there
gives you a different connection to the material, I think,
than other mediums have,
where there is a bit of a separation created by that,
you know, box, the frame.
So the frame is gone,
and I should clarify that we're not thinking,
"Oh, here comes virtual reality, so we don't write stories now,"
or something or, "We don't take photographs."
It's a new angle that we can give
a new kind of perspective on the story.
We created a virtual reality trip to Pluto
that we did in collaboration with NASA.
So this was just the most amazing material.
They, you know, I think in 2004, somewhere around that,
they launched the New Horizon spacecraft
which then traveled for three billion miles,
10 years, finally just got within 1,000 miles of Pluto
and took photographs, and we used that, those photos
and that data that NASA collected,
to give you a story about the experience
where you're kind of out there,
waiting for New Horizons to arrive,
and then you travel with New Horizons to the planet,
and then we actually put you down,
standing on the planet itself, based on accurate extrapolations
that we built together with NASA.
I mean, it's just such an engaging experience,
such a better way to engage an audience,
I think, in this material.
I think there's an interesting level,
maybe you could even say, with transparency
since this is a journalistic context.
Even with the best intentions, when you take a photograph,
or you're filming something,
you're making a crop.
So you're making all of these, you know,
at-the-moment editing decisions
of what you're including and excluding from the frame,
and you could argue that giving someone,
you know, a virtual reality view
or a 360 view or an omnidirectional view,
any way you want to kind of describe it,
it gets rid of the crop.
So you're giving to your viewer, "Here is the whole scene,
unedited in a sense,"
from, of course, from a very particular perspective.
Can't get around that, but it's still giving --
It's giving a little bit more, I think, to the viewer.
Another example was an attempt to
kind of use the power of the Times' archives,
and this was around the piece that we did
for the Rio Olympics, actually.
It was called "The Modern Games,"
and the concept was, "What if we could use...
Alright. What can we do with a virtual reality in the Olympics?
Well, what if we could recreate
the most inspiring moments
in the Olympic games throughout history,
since the beginning of the modern Olympics in 1896,
and let you stand there?"
It was based on all these incredible photos
that were in our archives
of, like, you know, Babe Didrikson in 1932
throwing a javelin,
and we turned these archival photos
into environments that you could stand in.
So when Babe is holding that javelin,
you're standing in the Los Angeles Colosseum
or whatever it was called in 1932, in front of her,
and now that photo is in full stereoscopic 3-D
as she's holding that javelin,
but it was all done with projection mapping
kind of techniques with the photography,
so you're really still looking at the real photography
even when you're in that.
You know, one of the things I've liked about it
is that you're really in a single thing
It kind of swims up against the rest of media in a way,
where, like, I'm watching TV,
but I'm running my laptop on the Internet.
I'm also checking Twitter, and I'm also doing this.
And in virtual reality, you're like,
"No. You're going to do just this for eight minutes,
and you're going to be completely immersed
in this story,
and it's going to take over everything."
I think people also kind of crave that a little bit.
It's such a fragmented, crazy world now
where just this just information overload
that it's nice to just be concentrating on a single thing
for a little bit and being really,
you know, inside of it, having it take over everything.
You know, the written word is not going away, but it's --
There are so many more opportunities now
to convey information,
and the written word is not always
the most efficient or most effective way to do that.
You wouldn't write thousands of words
so that somebody knows exactly what a painting would look like.
You show them a photo of it.
You don't have to always tell.
You can just show,
and I think virtual reality is part of that.
It is sometimes the shortest line
between what we're trying to communicate
and understanding for people,
and, you know, it's still an open question
as to the best ways that that actually happens
because it's such a new medium,
but, you know, we're trying it out, discovering it.
Hammonds: It's a very exciting time
in the development of virtual reality.
Creators and developers are working hand-in-hand
to really push those boundaries and create great experience
that are totally immersive and totally transport you.
I really feel like the sky is the limit.
Bradbury: Every single different media that emerges,
it finds a new pathway of telling stories,
and, in VR, I think it's going to be the same thing.
I think, for the first time, you can actually take variables
from the person that's viewing the experience
and put them into the experience.
That's what I think is the power of it all.
The point is to transport you to a place,
to make you feel like you're there.
Then you could argue that the more agency
that you give to somebody once they're there,
the more that they're going to feel the power of that presence,
of being there.
Roosegaarde: To drag it into an experience beyond the numbers
or the fact sheets
is a way to trigger a curiosity
and to engage people in a more emotional or interactive way.
It's also, when I look at myself,
I change because of experiences, not just because of numbers.
So if we want to create a world
which we deal with climate change
or rising sea level,
these kind of experience might be a key factor
to create change.
And it's great to be part of that
and fuel that with new stories, yeah.