Magic Music Visuals

I've been looking a lot at the shaders at shadertoy.com and was wondering: theyre written in GLSL right? how does that differ to FFGL plugins, and how do they create entire 3d models with just a few lines of code? there's all that abstract procedural stuff, but there are also pianos, walking characters, buildings and so on. how does that even work? i'm quite the amateur: i know of 3d models and textures, that can be used in realtime applications like games, but they need to be made beforehand. then i know of procedural generation of 3d objects like a minecraft terrain, that is created on the fly based on some algorithms (like those 3d mandelbrots). dont understand those algorithms, but it makes still sense. i also know of raytracing and all that physical calculation stuff that can be done in realtime like shadows, reflections, etc.

that would explain those terrain shaders or cloud shaders or fractal shaders and such on shadertoy. but then there're these https://www.shadertoy.com/view/MsXGWr, https://www.shadertoy.com/view/ldsGWB, https://www.shadertoy.com/view/MdXGW2, https://www.shadertoy.com/view/ldl3zN which i cant grasp at all. how do you make specific forms like a car or a piano ? and i wont even start on animating those objects.. ^^ apparently exepct from images, no external sources are used, so they have to be created in those lines of code somehow.

from my point of knowledge you could maybe give coordinates for polygons and their vertices, but there doesnt seem to be something like this in the code.

or am i thinking in the wrong direction at all, and it's all fake and just imitates looking like actual objects?

also: those shaders are written in GLSL right? what exactly are those FFGL plugins then, is FFGL also a shader language?

if anyone knows more of these underlying principles, i'd be happy if you could enlighten me

or am i thinking in the wrong direction at all, and it's all fake and just imitates looking like actual objects?

That's exactly right. The Vertex shader is a quad at the resolution and everything else (fragment shader) is done per pixel with math wizardry and by sampling textures.

If you would like to learn some of the techniques used you should check out Iñigo Quilez's great writeups particularly the ones about raymarching

as for FFGL it's a framework for creating openGL modules (made easy as opengl is a state machine. Many FFGL plugins are themself just shaders that expose parameters via ffgl.

If you're interested in writing something like this and don't care about using these effects in anything but Magic, Eric has done a really great job with the MDK and it's much quicker to get running with.

from my point of knowledge you could maybe give coordinates for polygons and their vertices, but there doesnt seem to be something like this in the code.

If you look closely there are actually equations in the code which define coordinates for vertices. So it's exactly what Syd said: math wizardry .

is FFGL also a shader language?

FFGL isn't a language, it's a plugin specification. It tells a software developer how to create code that will interface with a host application. It's exactly the same concept as a plugin for Photoshop, or for your web browser, or any other application. See http://en.wikipedia.org/wiki/Plug-in_%28computing%29. FFGL plugins could theoretically be written in any language, but mostly they are written in C++. As Syd described, FFGL plugins might make use of shaders, or they might not. It just depends on what effect the developer wants.

The thing about GLSL code is that it runs on the GPU, so you get a huge speed advantage. Quote from NVidia's web site: "A CPU consists of a few cores optimized for sequential serial processing while a GPU has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously." Image processing can usually take advantage of massive parallelism, because each pixel can be treated independently. In fact, what's really interesting about GLSL pixel shaders, like those on Shadertoy, is that their entire purpose is to set the color of one pixel. The shader code is run once for every pixel in the target image, so for a 1280x720 image, the shader is run 921600 times! And that's just one video frame!

With that kind of speed, you can essentially write your own renderer from scratch, which is what all those complicated shaders do. They completely bypass all the existing facilities for creating 3D images, such as meshes, lighting, etc. It's a new way of thinking about graphics.

Also have a look at http://en.wikipedia.org/wiki/Shader for more information if you're curious.

Thanks for the clearup on ffgl.

I really am curious about those shaders. In your example those 900'000 times calculating the shader is that done parallel then? Each pixel at the same time?

You say they bypass meshes and stuff, but they still make vertices and polygons? But on the fly?

What would be interesting now: create your 3d model and animation externally and reverse engineer them into lines of code .

I assume glsl isnt probably the best language to learn first?

btw eric, are some of your effects also "just" glsl shaders? like the pixelation one for example.

I really am curious about those shaders. In your example those 900'000 times calculating the shader is that done parallel then? Each pixel at the same time?

Things like that depend very specifically on your graphics hardware. More expensive graphics cards have more processors, but the current top-of-the-line cards still only have a couple thousand processors. So only that many pixels can be done at once.

You say they bypass meshes and stuff, but they still make vertices and polygons? But on the fly?

I just meant that they bypass the traditional meshes you'd see in video games, 3D modeling programs, etc. In shaders, the pixels are calculated using temporary vertices and polygons, but they don't exist in memory like game characters or terrains do, for example. So yes, "on the fly" is a good way to describe it .

What would be interesting now: create your 3d model and animation externally and reverse engineer them into lines of code .

Definitely! I'm not smart enough to do that though.

I assume glsl isnt probably the best language to learn first?

GLSL is very similar to C. C can be difficult for beginners, but it's certainly not impossible. Usually C is taught in entry-level computer science classes. There are tons of good online resources if you want to investigate further.

btw eric, are some of your effects also "just" glsl shaders? like the pixelation one for example.

That one isn't, but several others are, like the LumaKey. In a future version of Magic, all GLSL shaders will show up as modules like the LumaKey, so you won't have to deal with the vague X and Y params in the GLSLShader module.

In a future version of Magic, all GLSL shaders will show up as modules like the LumaKey, so you won't have to deal with the vague X and Y params in the GLSLShader module.

interesting, given any thoughts to implementation? ISF?

Yep, ISF was suggested to me a few months ago. So if it's not exactly ISF, it will be something very close.

Thanks a lot for the clearup eric. In that case i think i'll better stick to the high level lamguages first.theres still that c# intro ebook i should finish and im very interested in python. glsl here i come, in a few years

not what you asked for, but for this kind of stuff I can't recommend openframeworks enough. It's surprisingly easy to get into

http://openframeworks.cc/

looks very cool aswell. this is just an environment where one can write c++ stuff?