Magic Music Visuals

I'm trying to use billboard sprite techniques. Using any KEY in combination with DEPTHTEST reproduces the odd behaviour in the video, where the transparent area of one object seems to be rendering opaque.

The texture is a white circle on top of a green background.

It also exhibits a similar behaviour if I use a transparent PNG as the texture.

Is this a bug?

An excellent question. What you are touching on is a complicated, unsolved technical problem with real-time 3D graphics .

The short answer is that transparency and depth testing generally don't work together.

If you're feeling brave, you can read about it here: https://www.khronos.org/opengl/wiki/Tra ... cy_Sorting. But I'll try to explain it a bit more simply.

In almost all video programs (including Magic), the final frame is a result of all the composited layers. The compositing process renders the bottom layer first, then blends the next layer on top of that, and so on. The last thing drawn is the top layer. This is pretty standard behavior.

Consider this example:
Without the DepthTest module enabled, the lighthouse image is drawn on the bottom due to it being connected to the DepthTest module on the bottom, even though it is Translated in front of the desert image (closer to you in the z dimension).

With the DepthTest enabled, the lighthouse is drawn on top:
The DepthTest module's function is to "collect" everything that has been drawn up to that point, and get all the relative positions of things, and determine which is on top (closer to you). So it changes the draw order, kind of "overriding" Magic's connector order. Thus, the lighthouse is drawn last, even though its connector order would normally cause it to be drawn first (as the bottom layer of the final frame).

The problem is that, on all graphics cards, depth testing is a binary comparison (top-or-not), and it only deals with geometry (polygons), not textures. The power of graphics cards lies in their hardware-acceleration of drawing polygons, and for any point in space, polygons either exist or they don't -- there's no gray area. To the casual observer, it may simply look like there are two images in this scene, but the reality is that there are two rectangles (and actually, 4 triangles), which happen to be textured with images.

Now, consider this example:
I've added a Transparency module, so the lighthouse gets drawn with 50% transparency. But, the DepthTest module has determined that the lighthouse should be drawn on top because it is translated closer to you, and therefore your graphics card thinks the desert image is completely obscured. So the desert isn't drawn at all, and the lighthouse gets blended with nothing (since there's nothing else to be composited with in the final image).

It can be a bit tricky to get your head around it all, but the easy thing to remember is that you never want to have anything with partial transparency (such as your ChromaKey modules) connected to a DepthTest module. The DepthTest module simply won't take the transparency into account.

It is possible to have fancy algorithms that get around this problem, but as the page I linked to above says, "If you have enough translucent surfaces moving around in a sufficiently complex manner, you will find it very hard to avoid errors with acceptable realtime algorithms." This is exactly what's happening in your scene.

Also, so that you don't lose hope , I forgot to mention that the best solution is to use a program like Blender to apply textures to complex polygons. Then you can just load them into the Model module.

Something like this (you probably get the idea):
In this case, the "transparency" is created by the hole in the torus, which is 100% geometry, so it will work properly with depth testing.

It doesn't even have to be 3D, it could be a flat disk with a hole, or even just a circle, etc.

Eric got in before me but this is somewhat similar to this thread viewtopic.php?f=8&t=830

Thanks for that in-depth explanation Eric. It makes sense. (And for the model-based approach. It was where I would have gone next, actually making the geometry.) Sadly, I was originally hoping to do this with intersecting planes of green screen footage, but obviously that's a pipe dream.

I thought it might be because of the intersecting polygons, so I tried using 4 polygons with half-circle images where they were arranged in a cross pattern with no overlap in the geometry, where I thought there was no ambiguity in which polygon is in front. But it still failed. Now I get why.

I was modelling the approach on how grass is often rendered realtime in games, with thousands of transparent polys arranged in "stars" of 2 or three polys. They must be doing some fancy sorting.

I assume this is why so many volumetric effects are done via raymarching?

I was modelling the approach on how grass is often rendered realtime in games, with thousands of transparent polys arranged in "stars" of 2 or three polys. They must be doing some fancy sorting.

Could be. There are tons of ways to "cheat" on grass, which is what many games do. But theoretically you could still do it blade-by-blade without transparency if the polygons were actually shaped like blades.

I assume this is why so many volumetric effects are done via raymarching?

Yes, ray marching uses a completely different rendering technique and thus it isn't susceptible to the depth testing problem. But it is also usually much slower.