Using Kinect with Magic

[klangchaos]

Hello

my name is David.
I would like to use the Kinect as input similar to a webcam.
I would like to take advantage of the movement of people and thereby influence the visualization.

Is that possible?
Do you have ideas?

Thanks David

[blackdot]

that would be pretty cool. afaik, the only way to get data into magic is either audio, video or midi. so i imagine if you could translate something from kinect into a midi value, you could probably get that into magic. e.g. the x and y coordinates of a tracked object, or the distance of an object to the camera. but that would mean we'd need some third party tool that does this.

[dj0le]

Hi David

I actually just finished a music video using a Kinect + Magic. I think it turned out really well (I'll post links to it when it's released). Although you can't directly access a Kinect from within Magic, you can use other software to get it into Magic.

To access the Kinect data, I used Processing 3 + the Kinect v2 library. This allows you to capture any of the Kinect functions in real time (skeleton tracking, facial recognition, the IR depth field, or even the webcam itself). OpenFrameworks is another option and has a lot more libraries (but requires a significant level of coding ability). All of this software is open source, so no cost other than the timesink of learning to use it.

Then using Spout, it's easy to send it into Magic and do real time effects on it. You need a pretty solid rig to accomplish this though. The v2 Kinect takes up nearly the entire USB 3.0 bandwidth plus a significant amount of ram and GPU; it's very system intensive. When you throw in Spout + Processing + Magic, you will really tax your computer so it needs to be stout.

Because of that, I ended up capturing the Kinect data as stand alone footage from Processing, and rendered it into a movie with alpha, which I then imported into Magic and added a bunch of effects. That freed up all my system for running the various magic scenes. Here are a couple screen captures of that footage in Magic:

Fullscreen_3_20_16__10_20_PM.jpg (74.25 KiB) Viewed 24521 times

Fullscreen_3_20_16__10_22_PM.jpg (78.52 KiB) Viewed 24521 times

If you have any specific questions, I would be happy to answer them.

[Magic]

i imagine if you could translate something from kinect into a midi value, you could probably get that into magic. e.g. the x and y coordinates of a tracked object, or the distance of an object to the camera. but that would mean we'd need some third party tool that does this.

Exactly what I was thinking . So I Googled "Kinect to MIDI", and a few interesting results came up. Here is the first one: https://kinectmidicontroller.codeplex.com/. It seems like it would let you use your hands to control things in Magic.

Because of that, I ended up capturing the Kinect data as stand alone footage from Processing, and rendered it into a movie with alpha, which I then imported into Magic and added a bunch of effects. That freed up all my system for running the various magic scenes. Here are a couple screen captures of that footage in Magic:

That's very cool! Thanks for sharing.

[klangchaos]

thank people you are faster than a prime order at amazon for kinect apdapter
Kinect to midi I had not thought of it. good idea
I will if I have to apdapter times go through the suggestions and post you a results.
THANK YOU

[SoundRehabAustria]

Thanks for sharing this, guys. This makes very interesting reading - need to dig into this

[Slavi]

Hi David

I actually just finished a music video using a Kinect + Magic. I think it turned out really well (I'll post links to it when it's released). Although you can't directly access a Kinect from within Magic, you can use other software to get it into Magic.

To access the Kinect data, I used Processing 3 + the Kinect v2 library. This allows you to capture any of the Kinect functions in real time (skeleton tracking, facial recognition, the IR depth field, or even the webcam itself). OpenFrameworks is another option and has a lot more libraries (but requires a significant level of coding ability). All of this software is open source, so no cost other than the timesink of learning to use it.

Then using Spout, it's easy to send it into Magic and do real time effects on it. You need a pretty solid rig to accomplish this though. The v2 Kinect takes up nearly the entire USB 3.0 bandwidth plus a significant amount of ram and GPU; it's very system intensive. When you throw in Spout + Processing + Magic, you will really tax your computer so it needs to be stout.

Because of that, I ended up capturing the Kinect data as stand alone footage from Processing, and rendered it into a movie with alpha, which I then imported into Magic and added a bunch of effects. That freed up all my system for running the various magic scenes. Here are a couple screen captures of that footage in Magic:

Fullscreen_3_20_16__10_20_PM.jpg

Fullscreen_3_20_16__10_22_PM.jpg

If you have any specific questions, I would be happy to answer them.

I really like this idea!!!

In order to avoid the huge hardware requirements "Spout + Processing + Magic", do you consider viable the idea to have:
- "Dedicated Stage Set Up" (a PC for Processing+Kinect+TCPSpout(http://www.dusxproductions.com/blog/sha ... -and-macs/) )
- VJ Set Up (separate laptop for TCPSprout + Kinect )

I am looking into the idea of having this "Dedicated Stage Set Up" as a "kincet capture device" and broadcast it to my Magic Laptop where I would the all the VJ work.

Does anyone have any better ideas for caputring realtime Kinect data for live Magic pefromances, that doesn't involve a NASA computer ?

Regards,
S.

[Magic]

I would check out the Kinect-to-MIDI utility I posted above: https://kinectmidicontroller.codeplex.com/

There may be other similar utilities out there as well.

[Slavi]

Hi,

Thank you for the Tips!

Here is my first Kinect + Magic Video:
https://www.youtube.com/watch?v=_26phvF ... jBNX39eFUR

Tell me what you think

Regards,
S

[tribeteknofolle]

There is another software for Kinect V1 http://www.ethnotekh.com/software/kinectar/
Cheers
Andrea

[damstraversaz]

I'm using visution mapio pro for sending depth image to magic via spout. Tested since one year with dancers on stage and it works really great with magic. Mapio is maybe one of the best solution , as you can use depth with distance range, users image for masking or allow you to use kinect data directly for mapping (for exemple using just the hands and arms etc). It rocks really solid and is easy to install with the windows driver. You can use kinect V2 but I did not test it , as I do not need this version for my use.

here is a quick demo made by the developer about the kinect features.

I'm actually finishing an installation for kids about "body painting"controlling texture and color directly from the sound as I'm playing musical saw during the performance. The paint of each user will be saved as picture ( using fraps screenshot function). I will try to make a video about this ( not really easy, as it use live kinect input for video and live sound ). There is a lot to do with magic and kinect !

[damstraversaz]

here is one exemple, using Magic, mapio, and a kinect V1 for the visuals.

This a digital art installation. The exchange between a musician and the public is at the heart of this creation.

People and dancers create the trace , I'm playing a musical saw live, and use dynamic enveloppe and pitch tracking for controlling colors and opacity ( via midi input). For kids, it's possible to take a screen capture ( via midi foot controller) with fraps : each kid can keep a trace of his paint.
here is a live capture :

http://www.damientraversaz.com/traces

[Magic]

That's really nice! Thanks for sharing.

[Slavi]

Hi,

Absolutelly great and super inspiring !

Thank you for sharing!

Regards,
S.

[damstraversaz]

thanks !
I forgot: I use it since 3 months and it rocks solid. That's really important on live digital installation of course. I never had a problem or issue with the kinect V1 and this softwares ( generaly , traces is active for a session of 8-9 hours).

[damstraversaz]

some other exemples for a specific project with a dancer. I want to create a sort of "ink shadow", who works as a mirror of the dancer, so there is a sort of dialogue between the shadow and the real body.
Black and white works is really interesting, as you can use it with light on stage and still see it with a good definition ( tested with 3000 lumens projector short throw on stage)

kinect and magic

Traces encre 1.jpg (133.05 KiB) Viewed 23643 times

kinect and magic

traces encre 2.jpg (152.44 KiB) Viewed 23643 times

kinect and magic

traces encre 3.jpg (102.7 KiB) Viewed 23643 times

[walmartian]

Hi David,
You could try using http://www.visikord.com, it is based off the old milkdrop visualization software and allows you to use a Kinect (360 or One) and route live to Magic via Spout. He just released a free version that should be enough for your needs,although the paid version has some cool add-ons.
The interface is quite clunky but it performs well and does not require a supercomputer to run.
+1 for Mapio which has built in projection and pixel mapping.
Good luck,
David.