I've been intrigued by this question of
whether we could evolve or develop a sixth sense
A sense that would give us seamless access
and easy access to meta information or information that may exist somewhere
that may be relevant to help us make the right decision about whatever it is
that we're coming across and some of you may argue, well,
don't today's cell phones do that already?
But I would say no.
When you meet someone here at Ted and this
is the top Networking place of the year
you don't shake somebody's hand and then say
can you hold on for a moment, while I I take out my phone and Google you
Or when you go to the supermarket and
you're standing there in that huge isle
of different types of toilet papers you
don't take out your cell phone and open
a browser and go to a website to try to decide
to try to decide which of these toilet papers is the most ecologically
responsible purchase to make
So, we don't really have easy access to
all this relevant information that can just help us make optimal
decisions about what to do next and what actions to take
and so my research group
at the media lab has been developing a series of inventions to
give us access to this information
in a, sort of easy way without requiring
that the user changes any of their behavior
and I'm here to unveil our latest effort and most successful effort so far
which is still very much a work in process
I'm actually wearing the device right now
and, we've sort of cobbled it together with, components
that are, off the shelves and that by the way
only cost, $350 dollars at this point in time,
I'm wearing a camera, just a simple web cam
A portable battery powered projection system, with a little mirror.
The components communicate to my cell phone in my pocket which acts as the
communication and computation device
and in the video here we see my student
Pranav Mistry who's really the genius
who's been implementing and designing
this whole system and we see how this
system lets him walk up to any surface
and start using his hands to interact with
the information that is projected in front of him
the system tracks the four significant fingers
in this case he's wearing simple marker caps
I know you may recognize, but if you want
the more stylish version, you could also
paint your nails in different colors the camera basically tracks
these four fingers and recognizes any gestures
that he's making so he can just go to, for
example a map of long beach zoom
in and out etc, the system also recognizes
iconic gestures such as the take a picture gesture
and then takes a picture of whatever is in front of you
and when he then walks back to the media lab
he can just go up to any wall and project all
the pictures that he's taken sort through them
and organize them and resize them etc
again using all natural, ah gestures
so, some of you most likely were here two years ago
and saw the demo by, Jeff Han or some
or some of you might think doesn't this looks like the
microsoft service table and yes, you also interact
using natural gestures both hands etc.
but the difference here is that you
can use any surface. you can walk up to any surface.
including your hand if nothing else is available
and interact with this projected data
the device is completely portable
and can be, (laughs) thanks (aplause)
so one importance difference is that
it is totally mobile another even more important
difference is that in mass production
this would not cost more tomorrow than today's
cell phones and would actually not, um, sort of be a bigger
package and could look a lot more stylish
than this ah, ah version that (laughs) I am wearing around my neck
um, but other than letting some of you live out your
fantasy of looking as cool asTom Cruise in Minority Report
um the reason why we're really excited about this device
is that it um really can act as one of these sixth sense devices
that gives you relevant information
about uh whatever is in front of you, so we see Pranav
here going into the super market and he's ah shopping
for some paper towels and as he picks up
a product the system can recognize a product
that he is picking up using either image recognition
or marker technology and give him the green light or an orange light.
Um, he can ask for additional information
so this particular um (Audience Aplause) (laughs) choice here is a particularly
good choice given his personal criteria
some of you may want the toilet paper
with the most bleach in it rather than the most ecologically
responsible choice (audience laughter) um but (laughs) if
he picks up a book in the bookstore he can get
the Amazon rating, that gets projected right on
the cover of the book this is, uh Juan's book
our previous speaker um which gets
a great rating by the way on Amazon and so Pranav
turns the page of the book and can then see additional
information about the book, reader comments
um, maybe, sort of information by his favorite
critic etc. If he turns to a particular page
he finds an annotation by maybe an
expert of a friend of ours that gives
him a little bit of additional information
about whatever is on that particular page.
Reading the news paper he never has
to be (audience laughs) updated (laughs). You can get video annotations
of the events you are reading about. You can get
the latest sports scores etc.
This is a more controversial one (audience laughs)
As you interact with someone at TED
maybe um you can see a word cloud of the tags
the words that are associated with the person
in their blog and personal web pages
in this case the student is interested in cameras
etc. on your way to the airport if you
pick up your boarding pass it can
tell you that your flight is delayed
that the gate has changed etc.
And if you need to know what
the current time is, it's as simple as
drawing a watch (audience laughs) (laughs) on your arm
So, that's where we are at so far
in uh (laughs) developing this sixth sense that would
give us seamless access to all this
relevant information. About the things
that we may come across
my student Pranav who's really, like I said
the genius behind this (roaring applause)
(continued applause)
he does deserve a lot of applause because
I don't think he slept much in the last three
months actually and his girlfiriend
is probably not very happy
about him either
um, but it's not perfect yet
it's very much a work in progress
and who knows maybe in another ten years
we'll be here with the ultimate um
sixth sense brain implant, thank you.
(audience applause)