Integrate MMV with Unity 3D

Tue Mar 03, 2020 4:12 am

I want to be able to create audio/MIDI/OSC reactive 3D scenes. MMV doesn't seem to have a robust 3D editor nor out of the box lighting or PBR material workflows compared to something like Unity 3D. I tried making my own set of tools for Unity 3D (https://github.com/Cobertos/VisualsSandbox) but it was quickly unmaintainable and had a lot of problems with latency. MMV does have really nice workflows for processing MIDI, audio, etc. though and it would be nice if I could pass it to Unity to animate with and then do post processing on the final render back in MMV

A couple questions:

* What sort of 3D capabilities are there within MMV? Is there a workflow that you guys have found helpful for 3D stuff?
* Is there any way to send arbitrary values from MMV into an external program? I see that MMV supports Spout sending and receiving, but that only transmits images. I suppose I could encode what I wanted to send into an image, send it, and then decode in Unity, but if there's a more structured way that would be ideal. Plus trying to make everything sync up sounds... unideal.
cobertos
 
Posts: 2
Joined: Tue Mar 03, 2020 3:15 am

Re: Integrate MMV with Unity 3D

Tue Mar 03, 2020 8:41 pm

MMV can handle a small range of textured 3d model file-types. As others have found, the most reliable for textured objects is collada DAE. Other than that, as you say, no PBR, HDR lighting, etc. Remember that MMV walks the line between frame-rate, latency and visual quality. That said, there are several apps out there that can manage real-time 3d - TouchDesigner (to some degree), Smode (again it can struggle quite quickly), there's a glTF plugin for Resolume now and MadMapper has 3d with some lighting. Eric has mentioned several times that 3D is one of the bigger areas the team is looking at for future versions.

With the new iterator one can do some pretty interesting mograph type stuff in 3d but it suffers from a lack of lighting and materials. I'd like to try some baked lighting from Blender into MMV but haven't had the time to work it out yet, though that will only work with relatively static scenes.

You can get arbitrary values into MMV via OSC (an now out in an experimental build - ask Eric).

Also I have briefly tested ShowCockpit with sending Ableton-Link into MMV via OSC. You never know, there might be a driver for Unity one day.
Sadler
 
Posts: 685
Joined: Sat Aug 02, 2014 7:10 pm
Location: London, UK

Re: Integrate MMV with Unity 3D

Thu Mar 05, 2020 6:32 am

Ah, I was playing with .objs and I wasn't able to get a texture onto one of them. I'll have to try and play with that again in .dae. That was one of the things I wanted to mess with.

I will definitely have to look into ShowCockpit though, it sounds like a really good tool to lean on. I'll go ahead and message Eric as well
cobertos
 
Posts: 2
Joined: Tue Mar 03, 2020 3:15 am

Re: Integrate MMV with Unity 3D

Thu Mar 05, 2020 5:57 pm

Interestingly, a few years ago we looked into redesigning Magic with Unity as the main engine for everything (graphics, UI, etc.). We thought it would be a good way to have much better 3D support. The problem was that the 2D UI support was really lacking and wouldn't be as good as what Magic has now -- especially in terms of the Editor window where all the modules and connectors are. These days, it seems this is still a common complaint from people who want to build more sophisticated UIs in Unity.

I think that using Magic's new upcoming OSC output capability will open up a lot of doors though, and might be a good solution for people wanting to create and control more sophisticated 3D graphics.
Eric
Site Admin
 
Posts: 2786
Joined: Wed Apr 09, 2014 9:28 pm

Re: Integrate MMV with Unity 3D

Thu Apr 09, 2020 9:55 pm

I think TouchDesigner is a perfect companion for bringing Magic into 3D. Here is a simple example using two Spout feeds from Magic (using the FFGL Spout plugins) into TD. One as a material on a 3D object and one as a 2D background. You could also use Spout feeds as projection maps of lights, or have multiple camera angles from TD fed back into Magic via Spout, and on and on... :geek: Both applications have full Audio/MIDI/OSC reactive capabilities...
Magic_and_TD_example_1.jpg
Magic_and_TD_example_1.jpg (275.37 KiB) Viewed 930 times
j4s0n
 
Posts: 8
Joined: Wed Sep 03, 2014 6:06 am

Return to General Discussion

© 2020 Color & Music, LLC • This web site is mobile-friendly