Recently, I decided to begin working on a multi touch project, inspired by many, particulary Jeff Han, Bridger Maxwell, and other NUIGroup members. Essentially, there are two approaches to multi touch: FTIR (frustrated total internal reflection), and DI (diffused illumination). These techniques are explained well in the NUIGroup Wiki, here. Before embarking on a rather large scale, expensive project, I wanted a smaller scale, basic, proof-of-concept type example to create. I found this in Seth Sandler's MTmini. The MTmini is essentially a very basic form of front DI, composed of no more than a webcam, cardboard box, glass/acrylic, and a few sheets of paper.

How it works: the webcam goes in the cardboard box, the glass on top. The webcam is plugged into your computer, and when you press down on the glass, your fingers create shadows. The webcam picks these up, relays it to software that can comprehend these blobs on the glass (for example, TouchLib), and then TouchLib sends the data to an application which uses the touch event.

It seemed simple enough, and after a quick trip to Home Depot, I had everything I needed, and literally, in 5 minutes, I was done with the hardware. Now, for the software-TouchLib. From the NUIGroup website, I downloaded the latest version of TouchLib (2.0). From this point on, I had a bit of trouble with various components. I ran config.bat, as the instructions said to, which was fine, but with the MTmini, with no display, it was rather difficult to calibrate. Anyways, I was ready to try to run some applications, but I found that it wasn't all that simple. Running TouchLib successfully requires among other things, .Net 2.0 framework (which in turn requires Windows XP SP2, which requires Windows Update 3.0, which requires Windows Genuine Advantage Tool....). After I was done with these downloads, I ran into some more trouble as I found that in order to run Flash-based apps, which the demos are, I needed FLOSC, and JRE 1.6. Several other multi touch users helped me out at nuigroup's irc channel (irc.freenode.net #nuigroup) to help me get to a working MTmini setup, so much thanks to them. Nuiman explains the need for FLOSC/OSC very well:

"1) The TUIO (OSC) data is SENT from Touchlib’s “osc.exe”.

2) The FLOSC gateway CONVERTS the OSC (UDP) data into XML (TCP)

3) The Flash client uses an XMLSocket to RECEIVE and parse the XML data then renders the cursors within your application."

This is necessary because Flash cannot understand UDP data.

Anyways, I downloaded a whole bunch of things (more details later), and I was able to run Flash apps-basically, I should've downloaded this package, and I wouldn't have had the problems I was having, because cerupcat, the user who designed MTmini, had a special package for those using it, as it needed certain filters. Two .bat files downloaded with that package (similar to the TouchLib files I downloaded earlier), took the place of the FLOSC and OSC applications that I previously needed to run to have Flash apps recognize touch events.

So....now, my MTmini is in pretty good shape, and I am trying to fine tune the calibration of it, as the touches are not very accurate right now, and I am also trying to find ways to improve the hardware-better lighting, better webcam, etc. Software, of course, is a big part of it, and I am starting to learn to program in C++ in order to program applications for my multi touch device. Eventually, I plan to move past MTmini, and design and build a larger-scale, more stable, preferably FTIR multi touch table. But for now, I plan to work more with the software, and do what I can with MTmini.

Pictures of my MTmini setup coming soon!

MediaApp 0.5Beta Preview

MediaApp 0.5Beta Test

MediaApp 0.5 Beta Test 2

MediaApp 0.5Beta Test3

The main issue with multitouch tables in general is latency. There is a time-lag between the movements made by the user and the response by the software.

The main chain consists of the following:

One of the possible response-killers is Flosc + XML sockets. Flosc translates the X/Y coordinates from TouchLib to XML and offers the result to Flahs via XML Sockets. Flosc, however, offers no intelligence. Every event regarding changes registered in TouchLib is immediately fired to Flash via XML Sockets. More events (more fingers on the table) equals more data packets fired to Flash.

Being dependent of Flash polling for data, XML Sockets are not immediate / zero time latency as well.

When brainstorming about this with Erik van der Pluijm we came up with the following setup:

The C# wrapper is directly connected to TouchLib and the Flash Active X component. Any event in TouchLib is passed to Flash with "no" loss of time.

In the next week or so, this solution will be implemented.

Cercando qua e là per la rete sull'argomento "multitouch" se ne trovano di ogni, ma la cosa molto interessante che ho scoperto, è l'esistenza del Nuigroup, ovvero una community degli appassionati di tutto ciò che è "touch"

Tra questi c'è Seth Sandler, che ha creato un semplicissimo tavolo multitouch utilizzando una webcam ed una plastica trasparente coperta con un foglio di carta; il progetto si chiama MTmini
Utilizzando le zone d'ombra create dai polpastrelli come aree individuabili dalla webcam, una libreria TouchLib, interpreta i muovimenti delle dita e permette l'interazione con il desktop.
La libreria è in fase di sviluppo e i limiti di questo Proof of concept sono evidenti, ma è splendido pensare quanto sia semplice costruire un proprio Surface spendendo praticamente zero!

Ed è per questo che ho anche io sperimentato il progetto MTmini: ecco il video:

Qui le info necessarie per realizzare il proprio Surface:

Thinking that was pretty awesome, I tried to build it at home yesterday with pretty crappy results. I do know why though:

1. Primarily - the camera I was using was a serious piece of junk (really bad resolution and I can only expect really slow data transfer rates). So I know I just need to find a better camera with video capabilities; but we do have two other cameras. I did check one of them - but it didn't have any webcam video capabilities (but does record video! I didn't think this was even possible!); the other I didn't pull out of the van to try it out.
2. Secondary - the touchlib library would crash my XP workstation on starting the mousedriver or the smoke programs. Not all the time, but most times. Odd, but I probably could work through these issues (or at least go through the forums related to touchlib)...

Anyway, I was really looking forward to showing the kids and wife the little multitouch... maybe sometime this week. If I get it working, I will post pictures of my fabulous work.

Passeando pelo Instructables, esbarrei nesse manual de como fazer uma mesa multi-touch caseira. A diferença dessa pra outras, é que essa você só precisa mesmo é de uma webcam. O resto é configuração e ajuste na sua iluminação ambiente com ajuda do software. Software esse que é desenvolvido pelo pessoal do NUI Group, uma equipe de mídia interativa que pesquisa e projeta interfaces em open-source. TouchLib é o nome do projeto em questão, que detecta realces no histograma da imagem da webcam e os transforma em pontos de cursores a serem usados em diversos aplicativos.

Eis o vídeo mostrando o processo:

Sim, depois de ver isso eu fui fazer a minha própria. Tava com uma webcam sobrando aqui, então montei uma versão mini dessa caixa com o que consegui achar. Não ficou a coisa mais bonita do mundo, mas aí configurei o programa e funcionou tranquilo !

E viva la experimentación !

I haven't made one myself yet, but I downloaded touchlib and I found it quite annoying that it used my built-in cam from my toshiba sattelite laptop however I plugged in another external one.

So, I did a quick fix to be able to use /dev/video1 ... oh yeah, I tried this under Linux, more specifically Ubuntu, because I was lazy to setup all the libs with headers and shits under Windows.

In the CvCaptureFilter.cpp right in the void CvCaptureFilter::setParamater( ... ) I added two more lines, so my version looks like this:

[sourcecode language='cpp']

void CvCaptureFilter::setParameter(const char *name,
const char *value)
{
if(!capture)
{
if(strcmp( name, "source" ) == 0)
{
strcpy(source, value);
if(strcmp( value, "cam" ) == 0)
{
capture = cvCaptureFromCAM( CV_CAP_ANY );
}
else if(strcmp( value, "cam1" ) == 0)
{
capture = cvCaptureFromCAM( 1 );
}
else
{
capture = cvCaptureFromAVI(value);
}
}
}
}

[/sourcecode]

With this done, i can just change in the "config.xml" the "source" to "cam1" and make use of my other camera device on /dev/video1 :)

That's all I need :D

A few other thoughts ... you can give an index from 0..99 to the cvCaptureFromCam( ... ) and then it will try to find the first available driver for that particular device, or if you give something >=100 then it will try to use that particular driver with the first available video device; CV_CAP_ANY = 100 .

เอาล่ะ...ดูคนอื่นเขาสร้างสรรค์กันมาเยอะแยะแล้ว วันหยุดอย่างนี้ใครว่างอยู่ อาจจะลอง

ลุกขึ้นมาทำอะไรเล่นสนุกๆ กันบ้างก็ได้นะคะ ซึ่งซีรับรองว่า Gadget ทำมือ หรือ DIY

ที่นำมาฝากเพื่อนๆ วันนี้ ต้องเป็นที่ชื่นชอบกันแน่ๆ เพราะมันคือ ระบบมัลติทัชบนโน้ตบุ๊ก

(หรือเดสก์ทอป)ที่รันวินโดวส์ค่ะ :p

สำหรับระบบมัลติทัชรุ่นประหยัดที่เราจะทำกันนั้น ก็ไม่มีอะไรยุ่งยากเลย วัสดุที่ต้องใช้ก็

มีแค่ เว็บแคม แผ่นแก้วใส หรืออะคริลิก กระดาษขาว 1แผ่นและก็กล่องกระดาษ หลังจาก

เตรียมของครบแล้ว ขั้นแรกติดกระดาษขาวเข้ากับแผ่นอะคริลิกใสด้วยสก็อตเทป จากนั้น

เจาะรูบริเวณบนกล่องกระดาษให้มีขนาดพอที่จะสอดคอนเน็คเตอร์ของเว็บแคมออกมา

ได้ ใส่เว็บแคมเข้าไปตรงกลางกล่องยึดด้วยสก็อตเทป ยึดฝาครอบกล่องให้ติดกัน แล้ว

นำแผ่นอะคริลิกที่ติดกระดาษเรียบร้อยแล้วมาประกบที่ด้านบนของกล่องแล้วติดด้วยเทป

ใสอีกที ก็เป็นอันเสร็จเรียบร้อยแล้วค่ะ

ขั้นตอนการใช้งานก็นำมัลติทัชกล่องกระดาษของเราเนี่ยไปต่อกับโน้ตบุ๊ก หรือพีซี ซึ่ง

ได้รับการติดตั้งซอฟต์แวร์ touchlib ไว้เรียบร้อยแล้ว คราวนี้ เพื่อนๆ ก็สามารถใช้ระบบ

มัลติทัชบนพีซีของตัวเองได้แล้วล่ะค่ะ

รายละเอียดของอุปกรณ์ที่ใช้ในคลิปมีดังนี้ค่ะ

• กล่องกระดาษ
• แผ่นใส(กระจก, อะคริลิก ฯลฯ)
• กระดาษ A4
• เว็บแคม หรือวิดีโอแคมเมร่า (ในคลิปใช้ Philips SPC900NC)
• ซอฟต์แวร์ Touchlib

ใครทดลองทำดูแล้ว ได้ผลอย่างไร เล่าสู่กันฟังบ้างนะคะ :)

some of the projects i have seen lately are papervision3d, wiiflash, eyepoke and touchevent and the new flash-applications for touchlib (thx to whitenoiseaudio )

We will soon try our first applications based on this scaffold.

stay tuned...