I've been intrigued by this question of

whether we could evolve or develop
a sixth sense

A sense that would give us seamless access

and easy access to meta information
or information that may exist somewhere

that may be relevant to help us make
the right decision about whatever it is

that we're coming across and
some of you may argue, well,

don't today's cell phones do that already?

But I would say no.

When you meet someone here at Ted and this

is the top Networking place of the year

you don't shake somebody's hand 
and then say

can you hold on for a moment, while I 
I take out my phone and Google you

Or when you go to the supermarket and

you're standing there in that huge isle

of different types of toilet papers you

don't take out your cell phone and open

a browser and go to a website 
to try to decide

to try to decide which of these 
toilet papers is the most ecologically

responsible purchase to make

So, we don't really have easy access to

all this relevant information that
can just help us make optimal

decisions about what to do next
and what actions to take

and so my research group

at the media lab has been developing
a series of inventions to

give us access to this information

in a, sort of easy way without requiring

that the user changes any 
of their behavior

and I'm here to unveil our latest effort
and most successful effort so far

which is still very much a work in process

I'm actually wearing the device right now

and, we've sort of cobbled it together 
with, components

that are, off the shelves 
and that by the way

only cost, $350 dollars at 
this point in time,

I'm wearing a camera, just 
a simple web cam

A portable battery powered
projection system, with a little mirror.

The components communicate to my cell 
phone in my pocket which acts as the

communication and computation device

and in the video here we see my student

Pranav Mistry who's really the genius

who's been implementing and designing

this whole system and we see how this

system lets him walk up to any surface

and start using his hands to interact with

the information that is projected
in front of him

the system tracks the four
significant fingers

in this case he's wearing 
simple marker caps

I know you may recognize, but if you want

the more stylish version, you could also

paint your nails in different colors
the camera basically tracks

these four fingers and 
recognizes any gestures

that he's making so he can just go to, for

example a map of long beach zoom

in and out etc, the system also recognizes

iconic gestures such as the 
take a picture gesture

and then takes a picture of 
whatever is in front of you

and when he then walks
back to the media lab

he can just go up to any 
wall and project all

the pictures that he's taken 
sort through them

and organize them and resize them etc

again using all natural, ah gestures

so, some of you most likely 
were here two years ago

and saw the demo by, Jeff Han or some

or some of you might think
doesn't this looks like the

microsoft service table and
yes, you also interact

using natural gestures both hands etc.

but the difference here is that you

can use any surface. you can 
walk up to any surface.

including your hand if 
nothing else is available

and interact with this projected data

the device is completely portable

and can be, (laughs) thanks (aplause)

so one importance difference is that

it is totally mobile another 
even more important

difference is that in mass production

this would not cost more tomorrow than today's

cell phones and would actually not, um, sort of be a bigger

package and could look a lot more stylish

than this ah, ah version that (laughs) I am wearing around my neck

um, but other than letting some of you live out your

fantasy of looking as cool asTom Cruise in Minority Report

um the reason why we're really excited about this device

is that it um really can act as one of these sixth sense devices

that gives you relevant information

about uh whatever is in front of you, so we see Pranav

here going into the super market and he's ah shopping

for some paper towels and as he picks up

a product the system can recognize a product

that he is picking up using either image recognition

or marker technology and give him the green light or an orange light.

Um, he can ask for additional information

so this particular um (Audience Aplause) (laughs) choice here is a particularly

good choice given his personal criteria

some of you may want the toilet paper

with the most bleach in it rather than the most ecologically

responsible choice (audience laughter) um but (laughs) if

he picks up a book in the bookstore he can get

the Amazon rating, that gets projected right on

the cover of the book this is, uh Juan's book

our previous speaker um which gets

a great rating by the way on Amazon and so Pranav

turns the page of the book and can then see additional

information about the book, reader comments

um, maybe, sort of information by his favorite

critic etc. If he turns to a particular page

he finds an annotation by maybe an

expert of a friend of ours that gives

him a little bit of additional information

about whatever is on that particular page.

Reading the news paper he never has

to be (audience laughs) updated (laughs). You can get video annotations

of the events you are reading about. You can get

the latest sports scores etc.

This is a more controversial one (audience laughs)

As you interact with someone at TED

maybe um you can see a word cloud of the tags

the words that are associated with the person

in their blog and personal web pages

in this case the student is interested in cameras

etc. on your way to the airport if you

pick up your boarding pass it can

tell you that your flight is delayed

that the gate has changed etc.

And if you need to know what

the current time is, it's as simple as

drawing a watch (audience laughs) (laughs) on your arm

So, that's where we are at so far

in uh (laughs) developing this sixth sense that would

give us seamless access to all this

relevant information. About the things

that we may come across

my student Pranav who's really, like I said

the genius behind this (roaring applause)

(continued applause)

he does deserve a lot of applause because

I don't think he slept much in the last three

months actually and his girlfiriend

is probably not very happy

about him either

um, but it's not perfect yet

it's very much a work in progress

and who knows maybe in another ten years

we'll be here with the ultimate um

sixth sense brain implant, thank you.

(audience applause)