Page 1 of 1

Audio input to the GLSLShader module

Posted: Wed Dec 17, 2014 10:24 pm
by Magic
EDIT: In Magic 1.6, audio input to the GLSLShader module works differently. See the next post below.
---

Some of you may have been wondering what the GLSLShader module's "Audio->iCh." parameter is for.
AudioInput.png
AudioInput.png (12.66 KiB) Viewed 25881 times
If you have a shader that supports audio input, such as https://www.shadertoy.com/view/ldXGzN, you can enable the Audio->iCh. parameter to pass Magic's audio data into the shader.

Most of the time, shaders will use iChannel0 for the audio input, so you can leave the "iCh. Index" parameter at 0. But sometimes, depending on the shader, you'll have to change this value to 1, 2, or 3. The way to find out is to look at the Shadertoy page and see which box at the bottom has the audio in it. For example, this one uses iChannel1: https://www.shadertoy.com/view/Xdl3z7. Let it run for a bit so you can see how the audio changes the effect.

Hope that helps clear things up a bit. Now go create some cool stuff! :)

Re: Audio input to the GLSLShader module

Posted: Wed May 13, 2015 4:35 pm
by Magic
In Magic 1.6, I've changed the way audio input works for the GLSLShader module. I've separated the audio into its own module, allowing for greater flexibility which I'll explain below.

There is a new module called AudioToImage (in the Special menu) whose sole purpose is to draw audio data by converting amplitude to brightness. Without any other modules in the scene, and just the Waveform parameter selected, it looks like this:
AudioToImage1.png
AudioToImage1.png (47.58 KiB) Viewed 25612 times
What you're looking at is a "top-down" view of the Waveform. The peaks are brighter, and the troughs are darker. It's a good way to accurately represent audio with image pixels. Personally, I think this is a cool effect by itself.

You can also select the Spectrum parameter, which draws the audio spectrum in a similar way, with louder frequencies represented by brighter lines:
AudioToImage2.png
AudioToImage2.png (41.92 KiB) Viewed 25612 times
Lower frequencies are on the left side, and higher frequencies are on the right side.

Ok, so what does all this mean? Well, shaders (on the graphics card) don't have a way of directly accessing audio data (on the sound card). So if you want a shader to do something fun with audio, you have to pass the audio as an image. And that's exactly what the AudioToImage module does.

Putting it all together, and using the original example from the first post (https://www.shadertoy.com/view/ldXGzN), you can do this:
AudioToImage3.png
AudioToImage3.png (103.41 KiB) Viewed 25612 times
Any shader from Shadertoy that needs audio input can be used that way. Just make sure to select both the Waveform and the Spectrum parameter.

Re: Audio input to the GLSLShader module

Posted: Wed May 13, 2015 4:55 pm
by Magic
One other thing worth mentioning is that ISF shader modules work a bit differently than the GLSLShader module when it comes to audio input.

In general, ISF modules need separate AudioToImage modules for Spectrum and Waveform, and not all of them use both.

For example, this one only needs the Spectrum parameter selected:
audioToFFT.png
audioToFFT.png (26.78 KiB) Viewed 25612 times
But this one needs both Spectrum and Waveform parameters selected:
audioToFFT2.png
audioToFFT2.png (109.04 KiB) Viewed 25612 times
Unfortunately, a lot of the ISF modules aren't well-documented, so sometimes it isn't clear exactly what is necessary. You might just have to play around with it a bit to get it right.

Feel free to post any questions.

Re: Audio input to the GLSLShader module

Posted: Mon Aug 24, 2015 10:18 pm
by tomwhite
Thanks for the info!!!
I allways asked myself how to use the inputs of the shaders.
I definately should have joined the forum earlier :-)