How volumetric video brings a new dimension to filmmaking

I love making movies.

Motion pictures have been in existence

for more than a hundred years.

Filmmaking hasn't changed
for the dimensional mindset.

Placing the camera in a scene
and pressing "record" hasn't changed.

Filmmaking is still a frontal experience,

and creating the film has the possibility

to follow the same direction
of the content creation.

We still stand in front of a flat image

watching the fiction.

There's nothing wrong with it.

I love watching movies
and going to the theaters.

The experiences can be
such emotional experiences.

The arts and crafts of emotional
experiences within a frame

can be so strong

to drive a stronger emotion.

The question we're asking is,

how the experience of motion pictures
can exist beyond the flat screen?

How can we start creating content

for the next generation
of content experiences?

Traditionally, when we imagine a scene,

we look at the frame and the composition.

We have to think about how we create
depth and parallax

using foreground, background elements
as the camera moves.

With the technology today
and devices of VR glasses,

AR glasses, smart devices,

allowing three dimensional
and full navigation in space,

we have the possibility
to enable audiences

to experience content
from multiple perspectives.

What we have to think about

is how we take this technology,

all the capabilities

enabling the experience
to move farther away inside the scene.

Now, we're not talking about video games
or computer-generated actors,

which looks tremendously realistic.

We're talking about real actors
and real performance,

performing onstage.

We have to start thinking

how we capture the actors

and how we capture the real scene

in order to immerse inside.

Now, we're familiar
with the 360-degree video,

where you place the camera
inside the scene

and you can create this beautiful
panoramic image all around you,

but from the same aspect,

filmmaking is still frontal.

In order to emerge fully inside the scene,

we will need to capture the light
from all the possible directions.

We will have to surround the scene
with an enormous amount of sensors,

with all possible capabilities
to capture the light,

and enable us to emerge
inside afterwards again.

Now, in this setup,

there's no more foreground or background

or a camera placed in space,

but hundreds of sensors

capturing the light

and capturing the motion
from all the possible directions.

With the new technological advancements,

we can start looking at 3D photography,

capturing the light
from multiple perspectives,

enabling us to reconstruct the object.

This is like photography in 3D space.

Now,

with these technological advancements,

we can record video

not just as a flat image

but as a volume.

This is what we call "volumetric video,"

and it is the capability to record
every action of the scene

as a full three-dimensional volume.

Now, what is a voxel?

A voxel is like a three-dimensional pixel,

but instead of being a flat image square,

staying light and colored,

it's like a three-dimensional cube

in space,

with x, y, and z positions.

This enable us to create
a full capture of the scene

from any perspective.

Now, this renders
a fully light-immersive scene

from multiple perspectives.

This capability requires an insane amount
of information to be processed.

We will have to capture the light
from an enormous amount of cameras

to create this information.

Now, in order to do such a thing,

we would need a setup that would host
a numerous amount of cameras

installed in a stage,

and a stage big enough in order
to fit a full cinematic experience.

Now, that sounds like a crazy idea,
but that's exactly what we did.

For the last three years,

we have been building
a huge volumetric camera chamber.

It's 10,000 square feet of a stage,

enabling to capture the action
from any location.

We have deployed hundreds of cameras,

sending a tremendous amount of information

to a huge data center

powered by Intel supercomputers.

The ability to have this 10,000 feet

enables us to fit any kind of action,

any kind of performance.

It is the size of
an average Broadway stage.

We call it Intel Studios,

and it's the largest volumetric
stage in the world.

with the objective
of enabling and exploring

the next generation of this
immersive media filmmaking.

Now, to test these ideas,

we were thinking about what we can do

as the first scene to try it out.

So we chose the Western scene.

We brought horses, set designers,

dirt, everything needed to create
the full scene of a Western.

But this time, there was no camera inside.

There was nothing really moving
besides all the cameras

installed outside.

The challenge of the actors was tremendous.

They have to perform a flawless action
visible from all the directions.

There's no possibility to hide a punch
or not show the action.

Everything is captured
and everything is seen.

The output of the capture --

this is our future capture --

opened our eyes for
the immense capabilities.

It was like a full 3D scan
of the entire scene.

We were able to move around
And travel in the space.

The thing about this,

it's not anymore about perceiving
the light emitted from a screen,

but now traveling inside the light,

traveling inside the scene.

This obviously opens possibilities
for an enormous amount

of storytelling and
methodologies of creation.

This is the possibilities
of your personal narrative,

the possibility of creating
your own story inside,

or maybe following other stories.

Let's take a look at
one of the last renders and see.

(Music)

What you're seeing here
is full volumetric video,

and there's no physical camera

in the scene.

We have the full control

of space and time.

(Music)

Now, again, no physical camera was here.

Everything was captured surrounding.

Now, this is very nice,

but what if we wanted to see the scene
maybe from the eyes of the horse?

Well, we can do that as well.

So what you're seeing right now
is the same action.

But this time, we're watching
exactly from the eyes of the horse.

The possibilities are, well, unlimited.

(Applause)

Thank you.

So this is all great
for creators and storytellers.

It really opens a huge canvas

for a different type
of storytelling and moviemaking.

But what about the audience?

How can the audience
experience this differently?

In order to ?? our explorations,

we partnered with Paramount Pictures

in order to explore immersive media
in a Hollywood movie production.

Together with the director Randal Kleiser,

we reimagined the iconic movie of 1978,

"Grease."

Some of you know it, some of you don't,

40-year-old movie, amazing experience.

And our goal was really to look
how we can take this iconic action

and dance and bring it deeper
into the experience,

deeper into the audience.

Imagine that you can not just
watch the movie but get inside it

and dance with the actors,

and that's with the performance.

Now, we're breaking really the traditional
2D mindset of thinking and bringing

a much richer possibility of moviemaking
and content creation.

But why watch it on the screen?

Let's try to bring these actors
here on the stage.

So they're not going to really come.

I'm going to use --

(Laughter)

Sorry.

I'm going to use an iPad

in order to bring an augmented reality.

Now, obviously,

these devices have their own limitations

in terms of the data computing process,

so we have to reduce
the amount of resolution.

So what I'm doing now,
I'm placing here a marker

so I'll be able to position exactly
where I want everyone to appear.

OK.

I think we have them here.

(Applause)

Up. John Travolta, or a version of him.

Let's take a look.

(Music)

(Applause)

Thank you.

So as you can see,

we can watch and experience content
on the traditional way

or in an immersive way.

Really, the possibilities are open.

We're not trying to change
or replace movies.

We're enhancing them.

The technologies enable new possibilities

to start thinking beyond the flat screen.

We're in immersive and really
exciting times in filmmaking.

We're at the threshold of a new era.

We're opening the gates
for new possibilities

of immersive storytelling,

and exploration and defining
what immersive media filmmaking means.

We're really just at the beginning,

and we invite you all to join us.

Thank you.

(Applause)