Lab 3: 3D Transformations

Highlights of this lab:

This lab is an introduction to 3D Matrix Transformations.

Assignment

After the lab lecture, you have until the start of next lab to modify the files in Lab3.zip by:

Practicing modeling and viewing transformations with boxes.html and boxes.js then answering some questions
Creating controllable fingers for a robot arm based on robot_arm.html and robot_arm.js.

Seminar Notes

A. The Classic OpenGL Transformation Pipeline

The classic OpenGL pipeline had two main stages of vertex transformation, each with its own transformation matrix. These were built into the graphics hardware. These days, other transformation pipelines have become possible since transformations are done in the vertex shader. However, in this lab, as in the textbook, we will try to implement the classic pipeline.

Each vertex in the scene passes through two main stages of transformations:

Model view transformation (translation, rotation, and scaling of objects, 3D viewing transformation)

Projection (perspective or orthographic)

There is one global matrix internally for each of the two stage above:

M_modelview

M_projection

Given a 3D vertex of a polygon, P = [x, y, z, 1]^T, in homogeneous coordinates, applying the model view transformation matrix to it will yield a vertex in eye relative coordinates:

P’ = [x’, y’, z’, 1]^T = M_modelview*P.

By applying projection to P’, a 2D coordinate in homogeneous form is produced:

P” = [x”, y”, 1]^T = M_projection*P’.

The final coordinate [x”, y”] is in a normalized coordinate form and can be easily mapped to a location on the screen to be drawn.

Setting Up The Modelview and Projection Matrices in your shader

Since OpenGL Core Profile always uses shaders, neither the modelview nor the projection matrix is available. You have to set them up yourself. The matrices will be allocated and given their values in the main program, and they will be applied to vertices in the shader program.

To help us create and manipulate matrices in our main program we will use the matrix classes and helper functions in mat.h. Each matrix will be initialized to identity if you use the default constructor. So to create our initial modelview and projection matrices we would declare two MV.mat4 objects like so:
    
    let mv = new MV.mat4();   // create modelview matrix and set to identity
    let p = new MV.mat4();    // create projection matrix and set to identity
    
         
These two matrices can be modified either by assigning or post-multiplying transformation matrices on to them like this:
  

p  = MV.perspective(45.0, aspect, 0.1, 10.0); // Set the projection matrix to 
                                              // a perspective transformation 

mv = MV.mult( mv, MV.rotateY(45) ); // Rotate the modelview matrix by 45 degrees 
                                    // around the Y axis.

         
As in this example, we will usually set the projection matrix p by assignment, and accumulate transformations in the modelview matrix mv by post multiplying.

You will use uniforms to send your transformations to the vertex shader and apply them to incoming vertices. Last lab you did this for colours by making vector type uniforms and for point sizes by making a float uniform. Uniforms can also be matrices.
  
//other declarations
//...

//Uniform declarations
uniform mat4 mv; //declare modelview matrix in shader
uniform mat4 p;  //declare projection matrix in shader

void main()
{
  //other shader code
  //...

  //apply transformations to incoming points (vPosition)
  gl_Position = p * mv * vPosition; 

  //other shader code
  //...
}

         
To set the value of uniform shader variables you must first request their location like this:
  
//Global matrix variables
let projLoc;
let mvLoc;


//In your init code
// Get location of projection matrix in shader
projLoc = gl.getUniformLocation(program, "p");

// Get location of modelview matrix in shader
mvLoc = gl.getUniformLocation(program, "mv");

         
Then, you use a uniformMatrix* function with the uniform location and a local variable to set their value. Do this whenever you need to update a matrix - usually when the window is resized or right before you draw something. To set the value of our 4x4 float type matrices we will use the form uniformMatrix4fv:
// Copy mv to uniform value in shader
// in display routine, after applying transformations to mv
// and before drawing a new object:
gl.uniformMatrix4fv(mvLoc, gl.FALSE, MV.flatten(mv)); 

// Copy p to uniform value in shader
// after calculating a new projection matrix
// or as needed to achieve special effects
gl.uniformMatrix4fv(projLoc, gl.FALSE, MV.flatten(p)); 
 
         
Important: Notice that we use the MV.flatten() and not uofrGraphics'sflatten4x4() as you may have observed in Lab 1. If you switch you may find your scene breaks. If you prefer the slightly more performant flatten4x4() you will need to transpose the matrix either in JavaScript just before flattening, or in the vertex shader just before using it. The vertex shader in Lab 1's exercise does the latter.

Why? Most CPU oriented languages declare 2D arrays in Row Major order, but your GPU and GLSL expect matrices to be in Column Major. Some JavaScript math matrix libraries are written to match GLSL, but MV.mjs and the textbook code it is based on are not. In a full featured OpenGL, the second argument can be set to true to ask for a transpose, but the OpenGL ES and WebGL standards require that argument to be false. MV.flatten() has a built-in transpose step for matrices, but flatten4x4() does not.

B. Elementary Transformations

This section covers the 3D transformation matrix functions provided with your textbook. There are alternative libraries that come with their own in-depth discussion of similar transformations, such as Mozilla's Matrix math for the web article.

Defining a Coordinate System – Which Way Does Z-point?

Right-handed and left-handed coordinate system: With your right hand line your first two fingers up with the positive y axis and line your thumb up with the positive x axis. When you bend your remaining two fingers, the direction they point is the positive z axis in a right handed coordinate system. Compare to the figure below. The other system shown is a left-handed coordinate system. It is sometimes used in graphics texts. A consequence of using the right-handed system is that the negative z-axis goes into the screen instead of the positive as you might expect.
Right-handed coordinate system is used most often. In OpenGL, both the local coordinate system for object models (such as cube, sphere), and the camera coordinate system use a right-handed system.
In the following discussion, we assume that all transformation function calls return a matrix that you will post-multiply onto M_modelview, unless the other is specifically mentioned.
All transformation functions in this discussion that do not begin with gl. are equivalent or similar to a classic OpenGL transformation function and are defined in MV.mjs, but the MV has been omitted in function signatures. They all use the float data type for simple values. Appropriate vector types are indicated where needed.

Translation:

mat4 translate(dx, dy, dz)
Where [dx, dy, dz] is the translation vector.

The effect of calling this function is to create the translation matrix defined by the parameters [dx, dy, dz] which you should concatenate to the global model view matrix:

M_modelview = M_modelview * T(dx, dy, dz);

Where

In general, a new transformation matrix is always concatenated to the global matrix from the right. This is often called post-multiplication.
  
    mv = MV.mult( mv, MV.translate(0,0,-6) ); //Translate by -6 units on z-axis
  
         

Rotation:

There are two forms of rotation in MV.mjs.

Form 1
mat4 rotate(angle, vec3(x, y, z));
The first is similar to the only one available in classic OpenGL. It is capable of rotating by angle degrees about an arbitrary vector. However, it is often easier to rotate about only one of the major axes:

the x-axis: vec3(1,0,0)

the y-axis: vec3(0,1,0)

the z-axis: vec3(0,0,1)

These simple rotations are then concatenated to produce the arbitrary rotation desired. For example:
  
    // Rotate 20 degrees counter-clockwise around Y axis
    mv = MV.mult( mv, MV.rotate(20, MV.vec3(0,1,0)) ); 
  
         
Form 2

Rotating around only one axis at a time is so common that many matrix libraries provide special functions dedicated to each axis:
mat4 rotateX(angle)
mat4 rotateY(angle)
mat4 rotateZ(angle)

In the second form, angle is the angle of counter-clockwise rotation in degrees, and the axis is determined by the X, Y or Z in the function name.

Sample of the three forms of axis aligned rotation
Sample of the three forms of axis aligned rotation function.

The method for calling a rotation matrix is similar to translation. For example, this:
  
    mv = MV.mult( mv, MV.rotateX(a) );
  
         
will have the following effect:

M_modelview = M_modelview * R_x(a);

Where R_x(a) denotes the rotation matrix about the x-axis for degree a:

Applying rotation around the y-axis or z-axis can be achieved respectively by these function calls:

mv = MV.mult( mv, MV.rotateY(a) ); // rotation about the y-axis mv = MV.mult( mv, MV.rotateZ(a) ); // rotation about the z-axis

Scaling

mat4 scale(sx, sy, sz);
where sx, sy and sz are the scaling factors along each axis with respect to the local coordinate system of the model.

The scaling transformation allows a transformation matrix to change the dimensions of an object by shrinking or stretching along the major axes centered on the origin. The matrix for 3D scaling is defined as:

Example: to make the wire cube in this week's sample code three times as high, we can stretch it along the y-axis by a factor of 3 by using the following commands.

// make the y dimension 3 times larger mv = MV.mult( mv, MV.scale(1, 3, 1)); //Send mv to the shader gl.uniformMatrix4fv(mvLoc, gl.FALSE, MV.flatten(transpose(mv))); // draw the cube shapes.cube.draw();

It should be noted that the scaling is always about the origin along each dimension with the respective scaling factors. This means that if the object being scaled does not overlap the origin, it will move farther away if it is scaled up, and closer if it is scaled down.

The effect of concatenating the resulting matrix to the global model view matrix is similar to translation and rotation.

C. The Order of Transformations

When you post-multiply transformations as we are doing and as is done in classic OpenGL, the order in which the transformations are applied is the opposite of the order in which they appear in the program. In other words, the last transformation specified is the first one applied. This property is illustrated by the following examples.

The initial default position for the camera is at the origin, and the lens is looking into the negative z direction.

Most object models, such as cubes or spheres, are also defined at the origin with a unit size by default.

The purpose of model view transformation is to allow a user to re-orient and re-size these objects and place them at any desired location, and to simplify positioning them relative to one another.
Example: Suppose we want to rotate a cube 30 degrees and place it 5 units away from the camera for drawing. You might write the program intuitively as below:

// first rotate about the x axis by 30 degrees mv = MV.mult( mv, MV.rotateX(30)); // then translate back 5 mv = MV.mult( mv, MV.translate(0, 0, -5)); // Copy mv to the shader gl.uniformMatrix4fv(mvLoc, gl.FALSE, MV.flatten(transpose(mv))); // Draw a cube model centered at the origin shapes.wireCube.draw();

The following figure shows the effect of these transforms:

If you run this program, you might be surprised to find that nothing appears in the picture! Think about WHY.

If we modify the program slightly as below:

// first translate back 5 mv = MV.mult( mv, MV.translate(0, 0, -5) ); // then rotate about the x axis by 30 degrees mv = MV.mult( mv, MV.rotateX(30) ); // Copy mv to the shader gl.uniformMatrix4fv(mvLoc, gl.FALSE, MV.flatten(transpose(mv))); // Draw a cube modelcentered at the origin shapes.wireCube.draw();

The following figure shows the new result:

D. Modeling Transformation vs. Viewing Transformation

OpenGL uses concepts of a modeling transformation and a viewing transformation.

The modeling transformation is the product of the calculations for creating and laying out your model (making sure everything is correctly positioned and oriented relative to everything else in the model). The transformation functions scale(), rotate*() and translate() can be used to alter the modeling matrix.

The viewing transformation is the sequence of calculations for viewing the model (positioning the viewpoint so that you view the model from the orientation and position you desire). You could also use the combination of scale(), rotate*() and translate() for viewing transformations. The following discussion explains how this approach works. However, it involves the concepts of local and global coordinates and could be very confusing to some students. I would like to suggest students to skip this part first (notice I labeled it OPTIONAL), and proceed with the easy approach, lookAt(), discussed next.

OPTIONAL: Viewing With Regular Transforms

Consider the following code:
mv = MV.mult( mv, MV.translate(0, 0, -5) ); mv = MV.mult( mv, MV.rotateY(30) ); gl.uniformMatrix4fv(mvLoc, gl.FALSE, MV.flatten(mv)); gl.draw...(gl...., ..., ...); //Draw the model

Working down to the model's local coordinate system, we first move the local origin down the negative z-axis by 5 units and then rotate that coordinate system about the y-axis by 30 degrees.

Working up to the global coordinate system from the model, we first rotate the coordinate system about its origin by -30 degrees, then move it's origin down the positive z-axis by 5 units. The model is fixed, but the global coordinate system is rotated and translated. The viewpoint locates at the origin of the global coordinate system. Remember that in the global coordinate approach, the order is reversed, and the orientation order is also reversed.

The following picture illustrates the local approach to a rotation:

The following picture illustrates the global approach to a rotation:

Viewing with lookAt

mat4 lookAt (vec3 eye, vec3 at, vec3 up)
The lookAt() function defines a viewing transformation
Parameters

eye: specifies the position of the eye point

at: specifies the position of the reference point

up: specifies the direction of the up vector

The lookAt() function makes it easy to move both the "from" and the "to" points in a linear manner. For example, if you need to pan along the wall of a building located away from the origin and aligned along no axes in particular, you could simply take the "to" point to be one corner of the building and calculate the "from" as a constant distance from the "to" point. To pan along the building, just vary the "to" point.

The ModelView Matrix
You may be confused about the exact distinction between modelling and viewing. This confusion stems from the fact that we are imitating classic OpenGL which uses one matrix to represent both the modeling and viewing steps for everything that is drawn - the Modelview matrix. The transformation used to describe the model and the transformation used to describe the viewpoint's location and orientation coexist in that one matrix. This approach results in a simpler set of calculations in the graphics pipeline - it is much faster to calculate the modelview matrix once on the CPU than to potentially do it thousands of times in the shader program - once for every single vertex to be drawn. Matrix multiplication is not commutative but rather associative, which means that the product of ((AB)C) is the same as (A(BC)). Thus OpenGL's Modelview matrix is logically the product of a viewing matrix and a modeling matrix.

M_modelview = M_viewing * M_modeling

What this means is that your viewing transformations must be entered into the Modelview matrix before modeling transformations.

E. Viewport and Projection Transformations

Once you have learned Modelview transformations, the next step is to understand projection modes and viewport mapping.

Viewport Transformation

The gl.viewport() function is used to specify a viewport, or the drawable area in your WebGL canvas. It can be cause your draws to use all or only a portion of the canvas. It is best to call it at least as often as the canvas changes size and only after you know the size of the canvas. In WebGL, that means it could be in your init() function, but if you are using a variable sized canvas it would be best to call gl.viewport() as part of rendering.

gl.viewport(x, y, width, height)

x, y: specify the lower left corner of the viewport in canvas coordinates.

width, height: specify the width and height of the viewport in canvas coordinates.

Example:

// Correctly uses the size of the canvas as stored in the gl context // to set a full size viewport gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);

Projection Transformations

There are two basic methods of converting 3D images into 2D ones.

The first is orthographic, or parallel projection. You use this style of projection to maintain the scale of objects and their angles without regard to their apparent distance. MVnew.js provides ortho() to do this type of projection.

The second is Perspective projection. This is the most popular choice in 3D graphics. A perspective projection matrix can be created with the perspective() function.

Projection is handled by the M_Projection matrix. You do not usually concatenate to the projection matrix as you do with the modelview matrix.

Orthographic Projection

ortho()

mat4 ortho( left, right, bottom, top, near, far )

Parameters:

left, right: Specify the coordinates for the left and right vertical clipping planes;

bottom, top: Specify the coordinates for the bottom and top horizontal clipping planes;

near, far: Specify the distances to the near and far depth clipping planes. Both distances must be positive.

ortho() describes an orthographic projection matrix. (left, bottom, -near) and (right, top, -near) specify the points on the near clipping plane that are mapped to the lower left and upper right corners of the window, respectively, assuming that the eye is located at (0, 0, 0). -far specifies the location of the far clipping plane. Both near and far must be positive.

The following figure approximates an orthographic (actually it is for frustum() - see below) volume and the ortho() parameters

perspective()

In old OpenGL systems, a function with the same parameters as ortho() could create perspective transformations. It was called frustum() and though it was powerful, it was not very intuitive. There is a much simpler perspective command, called perspective(). Like frustum() it generates a perspective viewing volume but only a simple one. It lacks the flexibility of frustum() which can be manipulated to achieve special effects.

mat4 perspective( fovy, aspect, zNear, zFar )

Parameters:

fovy : Specifies the field of view angle, in degrees, in the y direction

aspect: Specifies the aspect ratio that determines the field of view in the x direction. The aspect ratio is the ratio of x (width) to y (height)

zNear: Specifies the distance from the viewer to the near clipping plane (always positive)

zFar: Specifies the distance from the viewer to the far clipping plane (always positive).

perspective() specifies a viewing frustum into the world coordinate system. In general, the aspect ratio in perspective() should match the aspect ratio of the associated viewport. For example, aspect=2.0 means the viewer's angle of view is twice as wide in x as it is in y. If the viewport is twice as wide as it is tall, it displays the image without distortion.

The following shows perspective viewing volume and the perspective() parameters

F. Saving and Restoring the Matrix

As you work, you will almost always need to either reset the matrix to the identity matrix, or save and restore a previous matrix state. To reset to the identity matrix use code like this:

mv = MV.mat4(); //restore mv to the identity matrix

To save and restore a matrix you can use a matrix stack. Classic OpenGL had one built in, but, like the rest of the matrix functions, it is missing in modern OpenGL varieties and must be provided by an external library. You can use any stack-like data structure that can handle your math library's matrix class. Javascript's arrays, which provide .push() and .pop() functions, are perfect.

To make a matrix stack in Javascript, write code like this:

Add to your global variables

//global modelview matrix stack let matStack = [];

In your render function or sub-functions, use .push() and .pop() around transforms that should only affect one or a limited set of objects.

matStack.push(mv); //Apply transforms to modelview matrix //Draw objects //... etc ... //restore old modelview matrix mv = matStack.pop();

You can store any mat4 matrix on the matrix stack so long as you remember to pop back to the correct matrices in the correct sequence.

Assignment

Goals of this assignment:
Master the use of the standard matrix transformations:

Viewing Transformations: through lookAt or equivalent modeling transformations

Projection Transformations: through perspective and ortho

Modeling Transformations: rotate*, translate, scale, and a matrix stack.

Part 1

Start with boxes.html and boxes.js from Lab3.zip.
As written, this program draws a basic coordinate system with a green x-axis, a red y-axis, and a blue z-axis. These will be referred to in the instructions as the axes

With the initial camera settings you are looking directly down the z-axis so you will not see it.

Make the following changes. Write your answers to the questions in steps 1, 2, 4 and 10.

Comment out the lookAt() call and replace it with a translate() with parameters ( 0, 0, -10 )
Is there any change in the display? Why? Why not?

Comment out both the lookAt() and translate() lines. What happens? Why?

Restore the lookAt() call.

Take a look at the perspective() call. The aspect ratio you were originally given was 1.0.

What happens when the aspect ratio is 1.0 and you change the canvas dimensions in boxes.html to width="512" height="256" ?

How about width="256" height="512" ?

The aspect ratio allows us to compensate for different shapes of canvas. Study the recommendations in point 2. of WebGL Anti-Patterns, then modify the aspect ratio in your perspective call so that it is an appropriate ratio of width to height based on the actual dimensions of the viewing area. Test the result with the two suggested canvas shapes to be sure you got it right.

Draw a wireCube centered at (0, 0, 0) relative to the axes. You can use the provided buffers and related shape data. Do this in the render function.

Move this cube so that it is centered at (1, 0, 0) relative to the axes.

Draw a second cube after the first - in a new colour if you can - and rotate it 45 degrees around the y-axis.

Place this rotated cube directly above the first cube. It will be centered at (1, 1, 0) relative to the axes. Be careful of the order of transformations.

The perspective view makes the two cubes look a little awkward. Try using orthographic projection instead of the perspective call. The function for that is: ortho. Use left, right, bottom, top, near and far values that include the whole scene and not much more. See the picture for expected results (some deviation is OK):

Please leave a commented perspective call in your program so I can see your answer to step 4c.

Rotate everything (using modeling transformations NOT lookAt) so that you are looking down at the top of the boxes and seeing the blue z-axis (and no red y-axis). See the picture for expected results:

If you wanted to leave your x and y axes unchanged, but still see the top of the boxes, like this:

how would you change your code?

Rotate everything so that you can see all three axes along with the two cubes. See the picture for expected results:

You may use different angles of course.

/5 marks

Part 2

Start with robot_arm.html and robot_arm.js from Lab3.zip.

First load the application and see how it works. Try pressing lower and uppercase 'e' to move the elbow. Try pressing lower and uppercase 's' to move the shoulder

Now, add three fingers and a thumb to the robot.
Use matStack.push() and matStack.pop() to separate the transformations for each digit. Do not attempt to "untransform" with an inverse rotate, translate or scale.

Finally, add some code that will make the finger and thumb move apart when 'f' is pressed and and together when 'F' is pressed. The center of rotation should be at the wrist.
Your completed robot hand might look something like the following.

You can interact with this sample solution to see how your arm might work. Click on it and use the keys described above. I have also added some additional controls:

r/R: to rotate the arm on the X axis so you can see it from different angles - most importantly from above

t/T: toggle between solid and wire cubes

p/P: toggle between perspective and ortho projections

/5 marks

Deliverables

Part 1

A working version of the program showing a result similar to that shown in step 11. Be sure to leave commented code where requested. Please provide your CS315 lab folder structure with at least your Lab3 and Common folders inside.
Document with written answers for the questions in Steps 1, 2, 4, and 10.

Part 2

A working version of the robot arm program with opening and closing claw and at least three fingers and one thumb. Please provide your CS315 lab folder structure with at least your Lab3 and Common folders inside.

References

Robot Arm Exercise: Neider, Jackie (1997) OpenGL Redbook, Chapter 3, USA: A-W Developers Press.

Selected pictures: Fosner, Ron (1997) OpenGL Programming for Windows 95 and Windows NT, USA: Addison-Wesley books.

WebGL2 Anti-Patterns: information on setting viewport and aspect ratio from canvas and gl context object properties.

OpenGL ES 3.0 Man Pages: detailed descriptions of OpenGL ES 3.0 functions, specifically uniformMatrix* and viewport.

Mozilla Developer Network docs: WebGL2RenderingContext: description of WebGL2 specific details of functions, specifically where they differ from OpenGL ES 3.0