Hi Magic team,
I’m a VJ and media artist in the Netherlands. Magic is my main ISF host, and I’ve built a library of 25+ multi-pass simulation shaders in it (Physarum, fluids, particle systems, reaction-diffusion) for live sets. Congratulations on v3. Native audio in ISF and multiple FFT inputs are exactly what I was hoping for.
Before I upgrade, I’d like to ask what v3 already covers, and share what would make it an easy decision for me.
Questions about v3
Which GLSL version do ISF shaders compile under? Can a shader declare #version 330?
Are PERSISTENT FLOAT buffers 32-bit now, or still 16-bit half-float?
Does the native ISF audio follow the ISF spec (“audio” and “audioFFT” inputs, with MAX for the bin count)? Can each input use a different audio source?
What exactly changed in the alpha handling? Will v2 projects composite differently?
Feature requests (most important first)
Modern GLSL for ISF. #version 330 would work on both Windows and macOS. It gives integer and bitwise math, texelFetch, textureSize, tanh/sinh and dynamic indexing. Today I need a workaround for each of these.
32-bit float persistent buffers, at least as an option. With half-floats, particle positions above ~1024 px snap to whole pixels, which creates visible lattice artifacts.
A wrap mode per pass: REPEAT for seamless (toroidal) simulations, and zero outside the texture (clamp to border). Edge clamping makes particle simulations gain mass at the borders.
IMG_SIZE() for pass targets, so buffer sizes don’t have to be hardcoded.
Intermediate passes as module outputs. One simulation could then feed several chains (for example, its trail map into a mask chain) without running twice.
Feedback between modules with a one-frame delay, so two separate shaders can react to each other. This is the core of my current work: layers that behave like living organisms and interact.
Per-module GPU time display. In long live sets I need to know which module is the expensive one.
Bugs I hit in v2.5 (please check if they’re still there)
The preprocessor replaces sampling macros like IMG_NORM_PIXEL even inside comments, which breaks compilation.
With about 28–30 inputs, the generated uniform declarations are cut off mid-line, and the shader fails with many syntax errors.
Requests 1–3 matter most to me. If they are in v3 or planned, I’ll upgrade right away. I’m happy to send small test shaders that reproduce each issue.
Thank you for keeping Magic alive and independent.
Best regards,
Yuri Gorbados
behance.net/gorbados
v3 upgrade — ISF questions and requests from a simulation shader artist
Re: v3 upgrade — ISF questions and requests from a simulation shader artist
Hi Yuri,
I'm not on the Magic team, but I've spent the last few months probing v3 with test shaders. Several of your questions have measured answers, so here they are while you wait for the official reply. Everything below was tested on Windows with an RTX 2080. I haven't tested macOS.
GLSL version: v3.0b6 already compiles ISF at 3.30 in the compatibility profile. A shader that prints `__VERSION__` reads 330, and gl_MaxTextureCoords still compiles, which it wouldn't under a core profile. I probed fourteen 1.30+ features and all of them compiled and returned correct values: texelFetch on a pass target, texture(), textureSize, textureLod, all five integer bitwise operators, uint, floatBitsToUint/uintBitsToFloat (bit-exact round trip), switch, dynamic indexing of a const array, integer % and /, isnan/isinf, round/trunc, and a non-constant loop bound. I didn't test tanh/sinh specifically, but they're part of 1.30. I also haven't tried declaring #version yourself. Since you already get 330, you shouldn't need to. Because it's the compatibility profile, gl_FragColor and texture2D keep working, so existing shaders don't break.
Persistent buffers: a FLOAT target is true 32-bit in v3. I wrote a precision ladder into a buffer and read it back. It gave a 23-bit mantissa and a smallest storable power of 2^-31, and it stored 2.0 and -0.5 unclamped. That should fix your 1024 px snapping. A target without FLOAT: true is still 8-bit and clamps to 0-1, so FLOAT has to be on any state buffer. Persistent buffers also start out cleared to transparent black, so first-frame reads are safe.
IMG_SIZE() on pass targets: this works now. I checked it against an independent measurement on a fixed-size target, an expression-sized target ($WIDTH/2.0), and a pass querying its own target, and all three were exact. Expression sizes like floor($WIDTH/3.0) allocate correctly too. One caution: an expression that mixed $WIDTH and $HEIGHT seemed to allocate a tiny target for me - inferred rather than probed, so I try to keep to one variable per expression - maybe I should test this again.
Wrap mode: as far as I can measure, pass targets are always LINEAR filtering with CLAMP_TO_EDGE, and there's no way to ask for REPEAT, NEAREST or border. Meanwhile I do toroidal sims with fract() on the coordinate, or texelFetch with integer mod, and get zero-outside by masking the read explicitly. Masking rather than just clamping also cured edge streaks I was getting in feedback buffers. If you read state with normalised coordinates under LINEAR, make the buffer dimensions powers of two so the texel centres are exact. Otherwise packed values get blended.
Native audio: in my tests audioFFT arrives as 1024 x 1, mono, at 48 kHz (not 44.1). That's 23.4375 Hz per bin, and to sample frequency f you need the half-texel offset: x = f/24000 + 0.5/1024. The values are already logarithmic, about (dBFS + 93)/93, so don't put another log curve on them. I always got 1024 bins and haven't tried MAX. The waveform "audio" input reported 0 x 0 for me, so I work entirely from the FFT. I haven't tested different sources per input - again more testing required.
Your two v2.5 bugs: IMG_ names inside comments have been safe since the September build, and compile errors now report absolute file line numbers, which makes debugging much easier. There is one related trap still live. A #define that takes a target as a parameter and contains an IMG_ call must fit on one physical line. With backslash continuations, the parameter name leaks into the generated uniform name and you get "undefined variable _SRC_imgRect". I haven't tested the 28–30 input truncation, so it would be worth sending them that repro.
Alpha, intermediate passes as outputs, inter-module feedback and per-module GPU time are challenging to test inside a shader and I don't think Magic supports it. I tried to implement passes as nodes a long time ago and didn't get very far. One partial workaround for sharing data though: I've passed structured data between modules by encoding it into an ordinary output image (a la TD image to point cloud hack), and it survives the trip intact.
Sadler
I'm not on the Magic team, but I've spent the last few months probing v3 with test shaders. Several of your questions have measured answers, so here they are while you wait for the official reply. Everything below was tested on Windows with an RTX 2080. I haven't tested macOS.
GLSL version: v3.0b6 already compiles ISF at 3.30 in the compatibility profile. A shader that prints `__VERSION__` reads 330, and gl_MaxTextureCoords still compiles, which it wouldn't under a core profile. I probed fourteen 1.30+ features and all of them compiled and returned correct values: texelFetch on a pass target, texture(), textureSize, textureLod, all five integer bitwise operators, uint, floatBitsToUint/uintBitsToFloat (bit-exact round trip), switch, dynamic indexing of a const array, integer % and /, isnan/isinf, round/trunc, and a non-constant loop bound. I didn't test tanh/sinh specifically, but they're part of 1.30. I also haven't tried declaring #version yourself. Since you already get 330, you shouldn't need to. Because it's the compatibility profile, gl_FragColor and texture2D keep working, so existing shaders don't break.
Persistent buffers: a FLOAT target is true 32-bit in v3. I wrote a precision ladder into a buffer and read it back. It gave a 23-bit mantissa and a smallest storable power of 2^-31, and it stored 2.0 and -0.5 unclamped. That should fix your 1024 px snapping. A target without FLOAT: true is still 8-bit and clamps to 0-1, so FLOAT has to be on any state buffer. Persistent buffers also start out cleared to transparent black, so first-frame reads are safe.
IMG_SIZE() on pass targets: this works now. I checked it against an independent measurement on a fixed-size target, an expression-sized target ($WIDTH/2.0), and a pass querying its own target, and all three were exact. Expression sizes like floor($WIDTH/3.0) allocate correctly too. One caution: an expression that mixed $WIDTH and $HEIGHT seemed to allocate a tiny target for me - inferred rather than probed, so I try to keep to one variable per expression - maybe I should test this again.
Wrap mode: as far as I can measure, pass targets are always LINEAR filtering with CLAMP_TO_EDGE, and there's no way to ask for REPEAT, NEAREST or border. Meanwhile I do toroidal sims with fract() on the coordinate, or texelFetch with integer mod, and get zero-outside by masking the read explicitly. Masking rather than just clamping also cured edge streaks I was getting in feedback buffers. If you read state with normalised coordinates under LINEAR, make the buffer dimensions powers of two so the texel centres are exact. Otherwise packed values get blended.
Native audio: in my tests audioFFT arrives as 1024 x 1, mono, at 48 kHz (not 44.1). That's 23.4375 Hz per bin, and to sample frequency f you need the half-texel offset: x = f/24000 + 0.5/1024. The values are already logarithmic, about (dBFS + 93)/93, so don't put another log curve on them. I always got 1024 bins and haven't tried MAX. The waveform "audio" input reported 0 x 0 for me, so I work entirely from the FFT. I haven't tested different sources per input - again more testing required.
Your two v2.5 bugs: IMG_ names inside comments have been safe since the September build, and compile errors now report absolute file line numbers, which makes debugging much easier. There is one related trap still live. A #define that takes a target as a parameter and contains an IMG_ call must fit on one physical line. With backslash continuations, the parameter name leaks into the generated uniform name and you get "undefined variable _SRC_imgRect". I haven't tested the 28–30 input truncation, so it would be worth sending them that repro.
Alpha, intermediate passes as outputs, inter-module feedback and per-module GPU time are challenging to test inside a shader and I don't think Magic supports it. I tried to implement passes as nodes a long time ago and didn't get very far. One partial workaround for sharing data though: I've passed structured data between modules by encoding it into an ordinary output image (a la TD image to point cloud hack), and it survives the trip intact.
Sadler
Re: v3 upgrade — ISF questions and requests from a simulation shader artist
Hi Yuri, thanks for your inquiry and thanks for using Magic in such an advanced way.
Sadler is correct about all the things he said for v3. For a bit more info:
- #version 330 is now the default, and you don't need to add it; it will be added for you.
- Yes, the ISF host implementation in Magic v3 now allows multiple separate audio and audioFFT inputs per shader, and you can even use separate audio input devices now.
- The alpha changes we made are for straight alpha vs premultiplied alpha. Previously, things were inconsistent, and some effects weren't handling alpha correctly, so we "normalized" everything to straight alpha. We edited all the included ISFs and they shouldn't cause any further problems, but if you're doing your own shaders, you may need to premultiply or unpremultiply if you notice things are too dark or too light.
- FFT bins are fixed at 1024 and currently not changeable (no one ever asked and it never really occurred to us), but maybe we can add it in the future.
- Same thing with the wrap mode.
- Intermediate passes as outputs / feedback / GPU timing: those would be pretty complicated to add now, but not impossible.
Let me know if you still have questions.
Sadler is correct about all the things he said for v3. For a bit more info:
- #version 330 is now the default, and you don't need to add it; it will be added for you.
- Yes, the ISF host implementation in Magic v3 now allows multiple separate audio and audioFFT inputs per shader, and you can even use separate audio input devices now.
- The alpha changes we made are for straight alpha vs premultiplied alpha. Previously, things were inconsistent, and some effects weren't handling alpha correctly, so we "normalized" everything to straight alpha. We edited all the included ISFs and they shouldn't cause any further problems, but if you're doing your own shaders, you may need to premultiply or unpremultiply if you notice things are too dark or too light.
- FFT bins are fixed at 1024 and currently not changeable (no one ever asked and it never really occurred to us), but maybe we can add it in the future.
- Same thing with the wrap mode.
- Intermediate passes as outputs / feedback / GPU timing: those would be pretty complicated to add now, but not impossible.
Let me know if you still have questions.
Re: v3 upgrade — ISF questions and requests from a simulation shader artist
Hi,
Thank you, and thanks to Sadler for the detailed tests. This answers almost everything, and it’s a clear reason for me to upgrade.
A few last questions:
Will MDK v2.3 modules work in v3, or is a new MDK planned?
Sadler saw the waveform “audio” input report 0×0. Is that expected, or a bug?
What does the upgrade from v2 cost, and is it available now?
I’ll test the 28–30 input limit in v3 and send a small repro if the problem is still there. A wrap mode option (REPEAT / NEAREST) and a configurable FFT size would still be welcome in the future.
Best regards,
Yuri
Thank you, and thanks to Sadler for the detailed tests. This answers almost everything, and it’s a clear reason for me to upgrade.
A few last questions:
Will MDK v2.3 modules work in v3, or is a new MDK planned?
Sadler saw the waveform “audio” input report 0×0. Is that expected, or a bug?
What does the upgrade from v2 cost, and is it available now?
I’ll test the 28–30 input limit in v3 and send a small repro if the problem is still there. A wrap mode option (REPEAT / NEAREST) and a configurable FFT size would still be welcome in the future.
Best regards,
Yuri