Where are the wacky mixtures of new technology that fundamentally change the way we interact with computers? A next generation cell phone is imagined as a phone with cooler buttons. A next generation computer is seen as just another computer you sit in front of except the buttons are shiny. It is time for a flight of fancy, something that is shockingly rare in the world outside of rarified blogs (and perhaps Nintendo’s hallucinogenic research labs.)
Here’s an idea called the Flashlight Computer that came to me this morning. It is a small portable computer that mixes camera technology, image processing, high intensity portable projectors and motion sensitive pointing devices to create a unique and intuitive human-computer interface.
Imagine a handheld computer that you point at a wall like a flashlight. A built in project illuminates a portion of your virtual workspace. At the center of the screen is your pointer. As you move your hand, the image naturally moves across the wall.
Now for the fun part. The image is constantly recalculated so that instead of moving with projector, it gives the illusion that it is painted onto the wall. As you move your computer, it acts as a flashlight, revealing new sections of your virtual workspace. A projector that displays a 3 square foot projection are turns a wall into a usable 80 square foot workspace.
Here’s how it works
- The projector displays registration marks on the wall.
- The device has a built in camera much like the Eye Toy that capture the registration marks at 120 FPS
- Image processing determines the perspective distortion of the image and alters the projector’s image so that it looks rectangular from the perspective of the user.
- As the user moves around, motion is tracked (either through image processing or internal gyroscopes) and the image on the wall is updated to make it appear as if the projector is revealing more of the workspace. To the user it appears as if you are panning across a single larger image. You literally illuminate your works space as you move.
Combine this with a simple onscreen pointer and a button for clicking and you have one powerful pointing device. It can be used anywhere, on any flat surface. You have full mouse capabilities including clicking, dragging and dropping, etc. It can be used to create a virtual world in a real space and it can also be used to augment the current world with virtual information.
What is it good for?
Here are some potential uses:
- Create the world’s largest continuous desktop. “Where did I leave that file? Oh, that’s right, I stored it in the other corner of the room.”
- Show walking directions at night. Shine the projector on the ground. It not only illuminates your path during the night, but it uses high resolution mapping information to paint a path that you can follow to your destination.
- Show complex wiring, pipes, etc inside a building by shining your flashlight at the wall and seeing the interior
- Games. There are entire universes of games that involve more than just sitting on your rump watching a mundane TV. For starters, just imagine playing an RTS title. Your entire house becomes a canvas.
Ideally, this control device is simple to use, under the user’s complete manual control and highly applicable to a wide range of applications. It would be easier to adopt than a head mounted display that is constantly attached to your noggin. It is much more flexible and convenient than being confined to a desk.
When will it be possible?
The technology to do this is a ways off. Portable projectors of the type needed are likely 20 years away. Computing power will get to the appropriate level in the next 5 years. Digital video cams are close as well. None of this technology is new. It just needs to mature and be integrated into a single system.
I can imagine that such a flashlight computer would be quite a thrill to use. 🙂
Take care
Danc.
LOSTGARDEN 
Cool idea. Will we see minimum requirements such as:3 rooms10ft high ceilings etcin the specs of these new games?
LikeLike
I can see a (FPS-like) horror game. Some sort of ghost-hunting thingy. The ghosts are coming for you, and you have to \”shoot\” at them.The scary part is that you see only part of the playing field (which turns to be your own home) at once. The part you can light with the torch. I think you get the idea.And… what about combining this with 3D? By using polarized light and glasses. I think it would not be much difficult to embed some sort of position-detection device in the glasses, so the projector knows how to cast the polarized beam.c u
LikeLike
WOW!!!i didn\’t know a HDS EDC light could do that.
LikeLike
At first, while reading this post I thought, \”this sounds fun but pointless,\” but then after the examples you gave I started thinking of a whole world of things you could do with these items.The first think that came to mind is repair tutorials. Imagine shining your computer at the engine of your car and it telling you where the parts is that you need to fix and how to fix it. Cooking. Painting. Anything we turn to the internet to get instruction on simplified by mapping a 3D space to a computer that then maps it to the actual object.Imagine virtual tours of art galleries where you are able to map the facility to your own walls or imagine viewing a blog of a friend where you can actually immerse yourself in the scene that they were in. If a picture is worth 1000 words, what is an entire room worth?
LikeLike
This comment has been removed by a blog administrator.
LikeLike
This is related, but interesting even if it weren\’t:PlayAnywhere Interactive Projector
LikeLike
This idea is fantastic, and the first paragraph of the post really resonated with me, we seem to be so focused on simply improving what we have without making any fundamental changes to its structure because of our fears of failure. I currently work as an Architect and the sheer magnitude of the building interior workings projection is astounding. Patent this idea: now. Props to you Danc, for continually proving that you\’re an ingenius thinker!
LikeLike
The most important question here is, what is the game that you\’re using screenshots from? :)X
LikeLike
This is a cool idea, but I see it being built out of several pre-existing ideas.1) The idea of integrated reality. Where cameras and computer processing create a virtual character that can navigate (and even interact with) a physical location.2) Small-scale laser based projection. Currently under development are projectors that would be small enough to fit in a cell phone but powerful enough to project a small image onto any surface.This is a cool idea. I like the idea of the navigation tools. I think that it has limitations because the computer needs to have pre-existing information about the surface to be able to project onto it. The program would have to know the size of a room, at least, to be able to scale the desktop.It resonates though, because of an idea of the computer as personal extension that I\’ve had for years. Imagine wearing a pair of glasses (not hard for me) that contains a pair of cameras for computer input and laser-based display capability to the surface of the glasses (or the retinas). With 3D image recognition, the whole world becomes the interface. You can pop-up information at any time.Combine it with facial recognition and a database of information about acquaintances and you will never forget the name (or personal information) of anyone again.Essentially computers without borders. No more \”window\” into the computer. Just digital extension of the world. It is a compelling vision and one that needs to be pursued.
LikeLike
Seconding Hunty\’s question, where do the graphics you used to illustrate the idea come from?Looks like a cool game.
LikeLike
All good ideas! Glad that the concept makes some sense. Ever have those ideas where you think to yourself \”That would be so cool!\” and then everyone looks at you like you are a lunatic. Happens all the time. 🙂 The game in the screenshot is Dofus Arena. I\’ve been looking at isometric tiles lately, and that one was in my inspiration folder. Danc.
LikeLike
the real challenge with any device is how it might improve productivity and become a necessity for EVERYBODY, not just a select few. The more uses the better chance a device has to succeed. SO add a cellphone, and make it as light as a feather and who knows… 🙂 –Ray
LikeLike
Dan – Your email is bouncing; Delay reason: mailbox is fullFor a few days now.
LikeLike
Problem with the email is now fixed. Thanks for the heads up! -Danc.
LikeLike
This is a really neat idea; I\’m a huge fan of augmented reality of most kinds, and I think this would be a great solution.Techically, there are certainly some hurdles. Beyond the obvious hardware problems, you\’d need the capacity to reduce lag to nearly zero, or the whole world would \”jitter.\” That sounds like either very careful tuning of the timing algorithms or an operating system dedicated to supporting the device (with dedicated realtime perspective-fixing algorithms that are non-interruptible). Perhaps the applications could paint the scene in a much-larger-than-screen buffer, and the OS could do a realtime read of that buffer. That system sounds like a very interesting challenge to create.
LikeLike
Saw something on Engadget that somewhat fits the bill. By somewhat, I mean loosely, but it\’s an actual concept involving a form of interactive projector, it looks like. The article has an image of the toy (drag draw), but not sure if there\’s any more information on it.Philips Simplicity Event 2006 shows off conceptual shiny toysSmall steps, right?
LikeLike
hey dan,whats that great rpgish image your useing as a sample? looks like something you made – can i have a look? :-)keep it up!
LikeLike