main-shaderantialiasing

Procedural Textures and Anti-Aliasing

Here’s a quick tutorial on doing procedural textures and anti-aliasing in GLSL.

I’ll make the simplest thing I can think of: a stipe down the middle of the object. I’ll do this by assigning one color to all the locations with texture coordinate u<.5, and another color to the other half of the object.

We’ll set the colors from our C++ program - OpenGL actually allows us to set 2 colors for each vertex:

    glColor3f(1,1,1);
    glSecondaryColor3f(.1f,.1f,.6f);

Assuming that you already know how to use a shader (see ShaderTools), lets write the simplest possible shaders to do this. We won’t handle lighting (yet).

The vertex shader just needs to pass the position, texture coordinates, and the two colors to the fragment shader:

// stripe0.vert
void main()
{
	gl_Position = ftransform();	// like this one
	// note: the variables are not passed to the fragment shader
	// by defult, gl parameters are not passed along - you need to do it
	// yourself
	gl_TexCoord[0] = gl_MultiTexCoord0;
	gl_FrontColor = gl_Color;
	gl_FrontSecondaryColor = gl_SecondaryColor;
}

One thing to notice: the output colors from the vertex shader are called “Front”.

The actual decision of which of the two colors happens in the fragment shader.

// stripe0.frag
void main()
{
	gl_FragColor = (gl_TexCoord[0].x < .5) ? gl_Color : gl_SecondaryColor;
}

Notice that this uses the ? operator (just like in C or C++), but since all of these quantities are vectors (which are native types in GLSL), its pretty clean.

Here is what it looks like in the sample program (which assigns texture coordinates inconsistently to the sides of the cube):

Notice that there is no lighting, and that there are jaggies.

Here is the same fragment program written a little bit differently.

void main()
{
	// note: I'll assign the color to a variable since
	// I might do lighting later
	vec4 baseColor;

	// pick the color based on the U coordinate
	float amt = step(.5,gl_TexCoord[0].x);
	baseColor = mix( gl_Color, gl_SecondaryColor, amt);

	// just use the color
	gl_FragColor = baseColor;
}

This looks exactly the same, but its written to be more extensible. There are 3 “good practices” here:

1. We’re putting the color into a variable (so we can change it or use it as part of a lighting computation later)
2. We’ve seperated the choice of which of the two colors (amt) as a seperate thing. This way if we wanted something more complex than a single stripe, we could do it.
3. We’re using the GLSL “mix” function to blend the two colors together. Right now, that’s just going to pick one or the other (since amt will be either 0 or 1), but in the future…

Now let’s get rid of the jaggies. The problem is since there’s that hard step, we get aliasing. What we need to do is blur the step a little bit so when we sample right at the edge we get a more gradual transition. Here’s a small change to do that:

void main()
{
	// note: I'll assign the color to a variable since
	// I might do lighting later
	vec4 baseColor;

	// pick the color based on the U coordinate
	float amt = smoothstep(.45,.55,gl_TexCoord[0].x);
	baseColor = mix( gl_Color, gl_SecondaryColor, amt);

	// just use the color
	gl_FragColor = baseColor;
}

Notice here I use the smooth step to have the stripe be a “soft edge”. The value will vary from 0 to 1 over the range around the place where I want the stripe to be.

This makes a blurry stripe. No jaggies, but blurry. Worse, since the width of the stripe is specified in texture coordinates, as we zoom in, it gets blurrier!

What we need to do to have a crisp stripe without jaggies is to set the width of the transition range in terms of the number of pixels, not necessarily a fixed amount in texture coordinates. To do this, we need to know how fast the texture coordinate is changing as we move from pixel to pixel. Fortunately, GLSL gives us easy functions for evaluating the derivatives which tell us this. We can check the derivatives in either direction with the dFdx and dfFdy function, or we can combine them together using the fwidth function.

void main()
{
	// note: I'll assign the color to a variable since
	// I might do lighting later
	vec4 baseColor;

	// pick the color based on the U coordinate
	float deriv = fwidth(gl_TexCoord[0].x);
	float amt = smoothstep(.5-deriv,.5+deriv,gl_TexCoord[0].x);
	baseColor = mix( gl_Color, gl_SecondaryColor, amt);

	// just use the color
	gl_FragColor = baseColor;
}

Notice how this gives us a stripe that stays crisp and jaggy free if we zoom in or out:

Of course, we still have no lighting, and we have jaggies at the edges of the triangles.