Magic Music Visuals

Hey everyone!
First of all — huge congratulations to Eric and the team on Magic 2.5 and the new AI prompting feature for ISF shaders! This is honestly a game-changer. I've been making AI ISF modules since the update dropped and it's amazing to have this built right into the workflow.
I wanted to bring up a feature request that I think could really benefit power users: the option to connect your own LLM API key — specifically something like Anthropic's Claude — as an alternative model in the AI shader settings.
Why I'm asking
I've been using Claude (via claude.ai subscription) to write ISF shaders for Magic for quite a long time now — way before 2.5 came out — and through dozens and dozens of sessions I've built up a really deep workflow with it. Here's a taste of what I've been able to create with Claude writing the GLSL code:

Fluid dynamics simulations — Navier-Stokes solvers, rotational CFD adapted from Flockaroo's Shadertoy work
Optical flow & datamosh feedback effects — the kind of glitchy video feedback that's normally a multi-pass nightmare
Delaunay mesh triangulation and Voronoi/foam cell systems
Plexus-style graph networks with animated connections
Volumetric cloud raymarchers
Curl noise tendril/smoke effects
Ribbed glass refraction and wave wobble deformation
Scribble portrait rendering and Saber-style glow effects
Sprite sheet animation tools with custom aesthetics

The critical thing is that Claude has learned (through our conversations) the specific constraints of Magic's ISF implementation — no persistent buffers between passes, no cyclic node connections, the difference between Shadertoy multipass and ISF multipass. When I describe an effect, it already knows to write single-pass procedural solutions or suggest manual feedback wiring workarounds specific to Magic. This context is incredibly valuable and took many sessions to build up.
The proposal
It would be fantastic if Magic 2.5+ could offer an "Advanced" or "Custom API" option in the AI settings where users can:

Paste their own API key (for Claude, or potentially other providers)
Select a model (e.g. Claude Opus, Sonnet, etc.)
Have Magic still handle the ISF-specific system prompt and format requirements on its side

This way, users who already pay for a Claude Pro subscription (or API access) could leverage their existing investment, potentially get access to more powerful models, and benefit from the provider's full context window for complex shader requests.
Benefits for the community

No extra cost for Magic team — the API calls go through the user's own account
Model choice — different LLMs have different strengths; Claude is particularly strong at GLSL/shader code in my experience
Longer context — Claude Opus has a massive context window, which really helps when you're iterating on complex shaders and want to reference previous attempts
Power user flexibility — casual users keep the built-in default (which is great!), advanced users get more control

For reference
I'm not the only one doing this kind of thing — over on the TroikaTronix (Isadora) forum, users have been sharing entire GLSL shader packs they created with Claude, including reference documents to make the AI more effective at writing shaders for their specific platform. The VJ/visual art community is clearly hungry for this kind of AI-assisted shader creation.
Would love to hear what Eric and the community think about this. Has anyone else been using external LLMs for shader writing? What models have you tried?
Cheers!

Thanks for your kind words. Just FYI, feature requests go here in the Feature Requests forum .

First of all, I'm not aware of any constraints with Magic's ISF implementation. For example, persistent buffers do work and are demonstrated in many of the included modules in the ISF menu (such as Optical Flow Distort, which is multi-pass, and has always worked well). But if there's something that's not working, please let me know and we'll fix it as soon as we can. You shouldn't need workarounds.

Second of all, using AI subscriptions inside of commercial apps is explicitly not allowed by the AI companies, and is not technically possible anyway. However, we actively encourage users to experiment with creating ISFs inside their subscriptions. That is one of the reasons we provided the ISFShader module in the latest release -- so that you don't have to restart Magic when creating/loading new shaders.

That being said though, some people don’t have or don’t want subscriptions, and it's definitely more convenient to work inside Magic and to see results immediately anyway. So, in terms of using the APIs (which is the only commercial option), there are many, many reasons why we came up with the system that we have. I'm happy to go into depth about technical issues or user experience issues if necessary, but suffice it to say that we think we came up with the best system for the most users. It "just works", and will continue to "just work" in the future as we add new options (some of which you might not even be able to access in other places). It's one payment, one access key, and all the issues are worked out by us so that you don't have to worry about them.

Also, you listed a lot of things that wouldn't make any difference whether it was your API key or ours; so I'm not sure I understand. For example, of course we allow you to select a model such as Claude... that is one of the primary features. Have you watched the tutorial video?

Overall, we definitely hope that this will be a benefit to the community. Creating static videos with Veo or Kling is one thing, but creating interactive video effects where you can control parameters in real-time is much more interesting to us. Our hope is that the models will get good enough, and GPUs will get fast enough, that they can render ever more beautiful, complex scenes that are fully interactive. Maybe even replacing static videos entirely someday!

Awesome - that shader list is really impressive, especially for single-pass. Curl noise, fluid dynamics, Delaunay mesh... not easy stuff. Post some pics?

A few things though:

• Magic already includes pro-level models in the built-in AI, so the quality bar is pretty high by default and means we don't have to pay for multiple (or even one) pro account.
• The real difference with using your own API isn't the model — it's the context. Magic's AI works with the Magic's ISF system context and the code from your current shader if you're iterating. Your private Claude sessions carry all that accumulated conversation history and your own specific knowledge. That's the part that's hard to replicate built-in. Perhaps offer your context to Magic devs to improve Magic?
• On multi-pass: ISF multi-pass shaders actually work fine in Magic, so Claude doesn't need to avoid them. Might be worth revisiting some of those effects with multiple passes — could open up even better results.

Also — you mentioned the Isadora/TroikaTronix community sharing reference documents for AI shader writing. Any chance you could drop a link? I'm sure folks here would appreciate.