my college project

[drslump]

Hi,

Most probably next year I'm gonna do a two-year course on network administration. It's not what I wanted to do but I can't go again to university, I don't feel like studying 5 years to get a diploma

So for this course they ask you to develop a project. I thought of one a couple of years ago but since it'll be quite time consuming I never started it. However I may implement it for the college.

Basicly what I wanted to do is to use a webcam as an input/pointing device. It means that you can point and click objects in the monitor using your hands, head or legs.
Since I don't think I can get much precission with this method it'd only be useful for entertainment apps, like games.

I don't want to reinvent the wheel, so does someone know about a similar project? I'm pretty sure someone has done it before 'cos the idea is pretty simple.

any tips will be welcome

cheers, Ivan

[J1NG]

You want to elaborate on that further?

All I got in my head after reading that was you want to use a webcam as an input pointing device. And a monitor which allows touch-screen techniques which may be used for games. But I can't see the link between them.

(This is good for you project, since you'll need to explain anyway )

J1NG

[Gurm]

This has been done many times.

Back in the mid 90's a friend got a PC development kit for the Nintendo Power Glove...

That used infrared, but this has been done before. Basically the camera would track, through a piece of software, your body movements, and use them to control the system.

Problem is that the software to process that image data is going to use nearly 100% of your cpu all the time.

- Gurm

[az]

No, no touch screen. The webcam tracking your fingertip.

That stuff is non-trivial, even with a reflecting dot on the fingertip.

AZ

[drslump]

No touch screen at all, as Az said all the input is from the webcam.

I suposse that it's going to be CPU intensive but perhaps I could use shader technology to manipulate the image, make it sharper, find the edges and that stuff. That should cut down the CPU usage drastically. And the project should be ready for 2005 so a 2ghz CPU should be common.

My first thought is that the way to follow will be to create a mask of the user body for every frame (5-10 fps). That would be the complicated issue. After that a simple per-pixel collision detection will do the rest.
The final objective will be to create an accurate mask of the user's body without using any reflective material or anything like that, just casual clothes. With the proper filters and a good callibration method that should be posible even with a random background (non high-contrast).


ivan

[Gurm]

The issue is that even in 2005 when everyone has a 2Ghz cpu... I'm willing to wager that this sort of algorithm would use MOST of a 2Ghz cpu's power.

- Gurm

[Wombat]

That's probably not a personal project, that's a research lab. If you want to see what you can do at home, Intel has released a pretty powerful image processing library you should investigate.

[DukeP]

ID say: GO for it!

Good coding can make anything possible.

If your not pressed against the impossible, you are neve going to make a miracle.

DIdnt they encode a simple optical meteor detection system on one of the first space thingies? Using less than 116 bytes??



~~DukeP~~

[dZeus]

similar concepts already exist for 'virtual keyboards'. Some Israeli firm developed and makes a match-box size device which will not only project keyboard outlines onto a surface it's standing on, it also has sensors which can measure where your fingers are, so when you touch the projected keys, it will correctly interpret which key you 'pressed'.

Dunno how it handles typing with 10 fingers very fast, where one finger may obstruct the view of the transmitter to another finger pressing a key

[drslump]

That Intel library sure looks pretty good, I must check its licence though.

Well, the point is that implementing such an algorithm, to recognize a body on a picture, will be very very slow to develop whitout using specific hardware.
The idea is to use a modern graphic chip (DX8 compatible) and develop the image transformation as fragment programs inside the GPU. Now, I've never done this, don't know if it's actually possible to do.

But then, I saw a program on TV the other day where a computer program could keep track of the players on a live football game. The developer said that the most difficult thing was to don't loose track of the players when making fast moves.
So, as I see it, you start the system in a specific position (callibration mode), then the program checks the pattern of the body. An human body cannot move in unlimited ways, I mean he can't be upsidedown (in theory) so it'll be easier to interpolate the coords of the extremities (arms, legs) between "key-frames".

I actually don't know what I'm talking about and that's what make me want to develop it
I could just make a simple web site with a search function and that would be ok for the teachers, but I preffer to do something that is a real challenge for me. After all I have two years to develop it.

ciao, ivan

[az]

Maybe you should implement this using a bright red spot on the finger (sticker, or thimble). Will make it way more easy than having to recognize the fingers.

Be sure to use a cam with a high frame rate.

BTW, there are head- and eye tracking devices out there, like the head mouse for disabled people.

AZ

[Fat Tone]

Head trackers can cost up to 10k USD. Eye trackers can cost a few times more!

I've done simple object following before (actually it was the moon). The image was divided into quadrants and I counted bright pixels in each quadrant to control a couple of tracking motors.

[az]

The Head Mouse costs ~2k, but that's with a red dot sticker. Some SLR cams have eye trackers, AFAIK, to determine the Autofocus point.

Following the moon seems way easier than following a complex pattern such as a finger. Following a bright red thimble might be easier.

AZ