Previous topicNext topic

LOD mipmap

Suggestions for new features for Magic.
Post Reply
Sadler
Posts: 1139
Joined: Sat Aug 02, 2014 7:10 pm
Location: London, UK

LOD mipmap

Post by Sadler »

I would like to be able to measure the average brightness of a frame and invert it if too bright or too dark. It sounded like it should be relatively easy so I investigated the possibility of a simple shader for this. There are quite a few ShaderToy examples that use level of detail on a texture call to quickly generate an average (or several other effects) but which require the input source to supply a mipmap filter. I don't understand much more than this but none of the ShaderToy examples that use a mipmap work in Magic.

A) Can anyone suggest another way to quickly find the average brightness of a frame?
B) mipmap seems to be efficient and useful for a range of effects. Could it be a new feature in Magic?
C) Does anyone have any ideas for a clever workaround for the effect I'm trying to achieve?
Terry Payman
Posts: 710
Joined: Sun Sep 14, 2014 8:15 am
Location: UK
Contact:

Re: LOD mipmap

Post by Terry Payman »

Is the Vidvox "Auto Levels" ISF shader any use as a starting point?

I don't really understand what it does, and don't have access to Magic at the moment, but I imagine that one could total the intensities of each input pixel in one pass, then output inverted or direct in a second pass. I think a persistent buffer or persistant float variable would be needed as an accumulator, but a persistent buffer certainly works in Magic. The decision would presumably be based on average luminance, easily calculated for each pixel before the accumulation.

However, I'm wondering if the invert decision would be extremely sensiitive to noise if the average value was close to 50%. An image containing many large light areas and many large dark areas could flicker very badly under the influence of a tiny bit of noise.

I suggest there needs to be some hysteresis added, for example 2%.
In the case of an image that one wished to keep dark, the output would be inverted if the average value was greater than 51%, but would only change back if the average dropped below 49%. It would be simple to add Hysteresis as a parameter in an ISF. Even so, large areas of the image could flick between very light and very dark when the threshold was crossed.
Terry Payman
Posts: 710
Joined: Sun Sep 14, 2014 8:15 am
Location: UK
Contact:

Re: LOD mipmap

Post by Terry Payman »

... A useful variant could be an option to constrain the luminance to a desired range, which would avoid the area flashes.
Sadler
Posts: 1139
Joined: Sat Aug 02, 2014 7:10 pm
Location: London, UK

Re: LOD mipmap

Post by Sadler »

Thanks Terry for the suggestion - I had a look into the auto levels but couldn't see how that could hint at a solution.

I have managed to create a working shader that does what I want, except that it samples a single pixel from a corner to test for brightness (> 0.9). As long as it's a primarily white or black frame the inversion is stable converting white backgrounds to black and keeping black backgrounds black. Perhaps this is good enough but I'd still like to find a way to average the frame pixels so that it can cope with a wider range of source material.

I think adding the brightness value of every pixel to find the average might be too much overhead but I won't know for sure until I try and this would surely depend on resolution. Access to the mipmap would provide this for free.
Sadler
Posts: 1139
Joined: Sat Aug 02, 2014 7:10 pm
Location: London, UK

Re: LOD mipmap

Post by Sadler »

Here's an ISF:
https://www.interactiveshaderformat.com/sketches/2945

Includes a point parameter with which you can adjust the pixel sample location and a threshold parameter to adjust the brightness value at which the invert is triggered. The sample point seems to work OK online but not so well in Magic - haven't work out why yet but perhaps something to do with normalised values. A bare GL shader works better in Magic using normalised mouse coordinates.
Magic
Site Admin
Posts: 3440
Joined: Wed Apr 09, 2014 9:28 pm

Re: LOD mipmap

Post by Magic »

Yes mipmaps would be a nice feature, though there is some overhead involved. But as you have discovered, you can use ISF to render to a very low-resolution intermediate image, let's say 3x3 pixels, and then just take the average value of those 9 pixels. This is pretty much what AutoLevels is doing.
Post Reply