Previous topicNext topic

Audio input to the GLSLShader module

Tutorials, FAQs, resources, and examples for using Magic.
Post Reply
Magic
Site Admin
Posts: 3440
Joined: Wed Apr 09, 2014 9:28 pm

Audio input to the GLSLShader module

Post by Magic »

EDIT: In Magic 1.6, audio input to the GLSLShader module works differently. See the next post below.
---

Some of you may have been wondering what the GLSLShader module's "Audio->iCh." parameter is for.
AudioInput.png
AudioInput.png (12.66 KiB) Viewed 25015 times
If you have a shader that supports audio input, such as https://www.shadertoy.com/view/ldXGzN, you can enable the Audio->iCh. parameter to pass Magic's audio data into the shader.

Most of the time, shaders will use iChannel0 for the audio input, so you can leave the "iCh. Index" parameter at 0. But sometimes, depending on the shader, you'll have to change this value to 1, 2, or 3. The way to find out is to look at the Shadertoy page and see which box at the bottom has the audio in it. For example, this one uses iChannel1: https://www.shadertoy.com/view/Xdl3z7. Let it run for a bit so you can see how the audio changes the effect.

Hope that helps clear things up a bit. Now go create some cool stuff! :)
Magic
Site Admin
Posts: 3440
Joined: Wed Apr 09, 2014 9:28 pm

Re: Audio input to the GLSLShader module

Post by Magic »

In Magic 1.6, I've changed the way audio input works for the GLSLShader module. I've separated the audio into its own module, allowing for greater flexibility which I'll explain below.

There is a new module called AudioToImage (in the Special menu) whose sole purpose is to draw audio data by converting amplitude to brightness. Without any other modules in the scene, and just the Waveform parameter selected, it looks like this:
AudioToImage1.png
AudioToImage1.png (47.58 KiB) Viewed 24746 times
What you're looking at is a "top-down" view of the Waveform. The peaks are brighter, and the troughs are darker. It's a good way to accurately represent audio with image pixels. Personally, I think this is a cool effect by itself.

You can also select the Spectrum parameter, which draws the audio spectrum in a similar way, with louder frequencies represented by brighter lines:
AudioToImage2.png
AudioToImage2.png (41.92 KiB) Viewed 24746 times
Lower frequencies are on the left side, and higher frequencies are on the right side.

Ok, so what does all this mean? Well, shaders (on the graphics card) don't have a way of directly accessing audio data (on the sound card). So if you want a shader to do something fun with audio, you have to pass the audio as an image. And that's exactly what the AudioToImage module does.

Putting it all together, and using the original example from the first post (https://www.shadertoy.com/view/ldXGzN), you can do this:
AudioToImage3.png
AudioToImage3.png (103.41 KiB) Viewed 24746 times
Any shader from Shadertoy that needs audio input can be used that way. Just make sure to select both the Waveform and the Spectrum parameter.
Magic
Site Admin
Posts: 3440
Joined: Wed Apr 09, 2014 9:28 pm

Re: Audio input to the GLSLShader module

Post by Magic »

One other thing worth mentioning is that ISF shader modules work a bit differently than the GLSLShader module when it comes to audio input.

In general, ISF modules need separate AudioToImage modules for Spectrum and Waveform, and not all of them use both.

For example, this one only needs the Spectrum parameter selected:
audioToFFT.png
audioToFFT.png (26.78 KiB) Viewed 24746 times
But this one needs both Spectrum and Waveform parameters selected:
audioToFFT2.png
audioToFFT2.png (109.04 KiB) Viewed 24746 times
Unfortunately, a lot of the ISF modules aren't well-documented, so sometimes it isn't clear exactly what is necessary. You might just have to play around with it a bit to get it right.

Feel free to post any questions.
tomwhite
Posts: 21
Joined: Mon Aug 24, 2015 10:03 pm

Re: Audio input to the GLSLShader module

Post by tomwhite »

Thanks for the info!!!
I allways asked myself how to use the inputs of the shaders.
I definately should have joined the forum earlier :-)
Post Reply