## DOT3 Versus Tangent-Space Bump-Mapping

One concept which has been used often in the design of Fragment Shaders and/or Materials, is “DOT3 Bump-Mapping”. The way in which this scheme works is rather straightforward. A Bump-Map, which is being provided as one (source) texture image out of several, does not define coloration, but rather relief, as a kind of Height-Map. And it must first be converted into a Normal-Map, which is a specially-formatted type of image, in which the Red, Green and Blue component channels for each texel are able to represent floating-point values from (-1.0 … +1.0) , even though each color channel is still only an assumed 8-bit pixel-value belonging to the image. There are several ways to do this, out of which one has been accepted as standard, but then the Red, Green and Blue channels represent a Normal-Vector and its X, Y, and Z components.

The problem arises in the design of simple shaders, that this technique offers two Normal-Vectors, because an original Normal-Vector was already provided, and interpolated from the Vertex-Normals. There are basically two ways to blend these Normal-Vectors into one: An easy way and a difficult way.

Using DOT3, the assumption is made that the Normal-Map is valid when its surface is facing the camera directly, but that the actual computation of its Normal-Vectors was never extremely accurate. What DOT3 does is to add the vectors, with one main caveat. We want the combined Normal-Vector to be accurate at the edges of a model, as seen from the camera-position, even though something has been added to the Vertex-Normal.

The way DOT3 solves this problem, is by setting the (Z) component of the Normal-Map to zero, before performing the addition, and to normalize the resulting sum, after the addition, so that we are left with a unit vector anyway.

On that assumption, the (X) and (Y) components of the Normal-Map can just as easily be computed as a differentiation of the Bump-Map, in two directions. If we want our Normal-Map to be more accurate than that, then we should also apply a more-accurate method of blending it with the Vertex-Normal, than DOT3.

And so there exists Tangent-Space Mapping. According to Tangent-Mapping, the Vertex-Normal is also associated with at least one tangent-vector, as defined in model space, and a bitangent-vector must either be computed by the Vertex Shader, or provided as part of the model definition, as part of the Vertex Array.

What the Fragment Shader must next do, after assuming that the Vertex- Normal, Tangent and Bitangent vectors correspond also to the Z, X and Y components of the Normal-Map, and after normalizing them, since anything interpolated from unit vectors cannot be assumed to have remained a unit vector, is to treat them as though they formed the columns of another matrix, IF Mapped Normal-Vectors multiplied by this texture matrix, are simply to be rotated in 3D, into View Space.

(Above Corrected 07/05/2018 . )

I suppose I should add, that these 3 vectors were part of the model definition, and needed to find their way into View Space, before building this matrix. If the rendering engine supplies one, this is where the Normal Matrix would come in – once per Vertex Shader invocation.

Ideally, the Fragment Shader would perform a complete Orthonormalization of the resulting matrix, but to do so also requires a lot of GPU work in the FS, and would therefore assume a very powerful graphics card. But an Orthonormalization will also ensure, that a Transposed Matrix does correspond to an Inverse Matrix. And the sense must be preserved, of whether we are converting from View Space to Tangent-Space, or from Tangent-Space into View Space.

## Self-Educating about Perpendicular Matrices with Complex Elements

One of the key reasons for which my class was taught Linear Algebra, including how to compute Eigenvalues and Eigenvectors of Matrices, was so that we could Diagonalize Symmetrical Matrices, in Real Numbers. What this did was to compute the ‘Perpendicular Matrix’ of a given matrix, in which each column was one of its Eigenvectors, and which was an example of an Orthogonal Matrix.  (It might be the case that what was once referred to as a Perpendicular Matrix, may now be referred to as the Orthogonal Basis of the given matrix,?)

(Edit 07/04/2018 :

In fact, what we were taught, is now referred to as The Eigendecomposition of a matrix. )

Having computed the perpendicular matrix P of M, it was known that the matrix product

PT M P = D,

which gives a Diagonal Matrix ‘D’. But, a key problem my Elementary Linear class was not taught to solve, was what to do if ‘M’ had complex Eigenvalues. In order to be taught that, we would need to have been taught in general, how to combine Linear Algebra with Complex Numbers. After that, the Eigenvectors could have been computed as easily as before, using Gauss-Jordan Elimination.

I have brushed up on this in my old Linear Algebra textbook, where the last chapter writes about Complex Numbers. Key facts which need to be understood about Complex Vector Spaces, is

• The Inner Product needs to be computed differently from before, in a way that borrows from the fact that complex numbers naturally have conjugates. It is now the sum, of each element of one vector, multiplied by the conjugate, of the corresponding element of the other vector.
• Orthogonal and Symmetrical Matrices are relatively unimportant with Complex Elements.
• A special operation is defined for matrices, called the Conjugate Transpose, A* .
• A Unitary Matrix now replaces the Orthogonal Matrix, such that A-1 = A* .
• A Hermitian Matrix now replaces the Symmetrical Matrix, such that A = A* , and the elements along the main diagonal are Real. Hermitian Matrices are also easy to recognize by inspection.
• Not only Hermitian Matrices can be diagonalized. They have a superset, known as Normal Matrices, such that A A* = A* A . Normal Matrices can be diagonalized.

This could all become important in Quantum Mechanics, considering the general issue known to exist, by which the bases that define how particles can interact, somehow need to be multiplied by complex numbers, to describe accurately, how particles do interact.