Computer Graphics
Jochen Lang
jlang@uottawa.ca
Faculté de génie | Faculty of Engineering
Jochen Lang, EECS jlang@uOttawa.ca
This Lecture
• Introduction to OpenGL, WebGL and Three.js
– Game or Rendering Engines, Graphics API, Low-level APIs
– Three.js, OpenGL (ES) and WebGL
– Programmable pipeline
– Drawing a triangle (yeah!)
Jochen Lang, EECS jlang@uOttawa.ca
Graphics API
• Different API levels
• Low-level APIs
– Goal: exposing features of the hardware
• Intermediate-level APIs
– Goal: hardware access but also support of rendering pipeline
• Game engines and other development platforms – Support specific task irrespective hardware
Jochen Lang, EECS jlang@uOttawa.ca
Low-level APIs
• Examples are Microsoft Direct3D 12 (Windows platforms only), Metal (Mac, iOS only), Vulkan (cross platform), WebGPU (work in progress)
– Provide access to the rendering hardware exposing hardware features
– Graphics card as stream processor
– Must use other libraries for other tasks, e.g., windowing, math etc.
– Must setup interactions with hardware
– Large resemblance with OpenGL or Direct3D 11 but increased flexibility
Image source: Wikimedia
Jochen Lang, EECS jlang@uOttawa.ca
Intermediate-level APIs
• Examples are OpenGL, OpenGL ES and WebGL or Direct3D 11
– Design mismatch of APIs to new hardware
– Lack flexibility as particular processing pipeline is assumed
– Easier to use than low-level APIs
– Large community and many tutorials
– No direct replacement
Image source: Wikimedia
Jochen Lang, EECS jlang@uOttawa.ca
General-purpose Graphics API for the Browser
• Examples are Three.js and Babylon.js for browsers
– Supporting one or more lower-level APIs
– Provide ready-made solution for typical tasks
– Easier to learn and faster success
– Not flexible, quirks may be hard to understand and change
See https://threejs.org/ for examples See https://www.babylonjs.com/ for examples
Jochen Lang, EECS jlang@uOttawa.ca
Game engines
• Examples for full-featured application development kits are Unity or Unreal
– Support multi-low-level APIs, e.g., “Unity supports DirectX, Metal, OpenGL, and Vulkan graphics APIs” as of 2020.2
– Locked-in to specific approach
– Features are limited by API
– Enable to concentrate on task at hand
– Other examples: Amazon lumberyard, OGRE, jMonkeyEngine
Image source: Wikimedia
Jochen Lang, EECS jlang@uOttawa.ca
Rasterization Rendering Paradigm
• Idea:
– Geometric primitives (points, lines, and triangles) are projected into 2D image space
– Pixels in image are tested whether they belong to the primitive or not
– If yes, the pixel’s colour is updated*)
• Current graphics hardware implements rasterization
*) Z-buffer test, blending etc. may prevent or change the update
Jochen Lang, EECS jlang@uOttawa.ca
Graphics Rasterization Pipeline
• Four major tasks:
1. Modelling (mostly a pre-process)
• Produces geometric primitives and description of surface, volume and light properties
2. Geometry processing
• Normalization (change to camera, canonical coordinates or normalized device coordinates), clipping, culling, hidden- surface removal and possibly shading
3. Rasterization
• Deciding on mapping of pixels to primitives, e.g., triangle rasterization and line drawing, followed by fragment processing
4. Display
• transfer of frame buffer to screen; may require, e.g., anti- aliasing and tone mapping
Jochen Lang, EECS jlang@uOttawa.ca
2D Triangle Example: 1. Modelling
– Define the vertices for the triangle – Define colours
– Decide on image borders
– Use model coordinates
Jochen Lang, EECS jlang@uOttawa.ca
2D Triangle Example: 2a. Geometric Processing
• Normalization
– Change the vertices for the triangle into normalized device coordinates
– Normalized device coordinates are and
Jochen Lang, EECS jlang@uOttawa.ca
2D Triangle Example: 2b. Geometric Processing
• Viewport Transform
– Map continuous normalized device coordinates to fragments (potential pixels)
– Fragment coordinates dependent on the number of pixels of the viewport (buffer, window)
– Here and
Jochen Lang, EECS jlang@uOttawa.ca
2D Triangle Example: 3. Rasterization
– Fragments in viewport are tested whether they belong to the primitive (here: triangle) or not
Jochen Lang, EECS jlang@uOttawa.ca
2D Triangle Example: 4. Transfer to Framebuffer
– Fragments in viewport are transferred to pixels on display device
Jochen Lang, EECS jlang@uOttawa.ca
WebGL
• Runs in your browser
– Supported by all major web browsers (including IE)
• Uses the HTML 5 canvas element
• Requires only basic programming experience
– Javascript is used to interface with OpenGL
• No fixed pipeline, only vertex and fragment shaders
• Based on OpenGL ES 2.0
Jochen Lang, EECS jlang@uOttawa.ca
Basic WebGL Program
function main() {
// Retrieve
Simplified OpenGL Pipeline
Geometry
Vertex Processor
Geometry Processing (Per- vertex operation)
Rasterization
Texturing (Per-fragment operations)
Textures and Images
Fragment Processor
Frame- buffer
Per- fragment operations
Jochen Lang, EECS jlang@eecs.uOttawa.ca
GL Operations
• Group in the three areas: – Geometry
• Different geometric primitive are available: points, lines, triangles and quads
– Texturing/Image primitive operations
• Bitmaps, e.g., glBitmap, Bitmapped fonts
• Images, pixels, e.g., glDrawPixels, glReadPixels • Texture mapping, e.g., glTexImage2D
– Set states of the OpenGL state machine
• Color, e.g., glClearColor, glColor
• Line styles, e.g., glLineStipple, glLineWidth
• Transformations, e.g., glMatrixMode, glMultMatrix
• Display modes, e.g., GL_LIGHTING with glEnable, glDisable • etc.
Jochen Lang, EECS jlang@uOttawa.ca
Using Shaders
• Why shaders?
– Immediate mode rendering with fixed pipeline:
• restrictive
• slower (?)
• harder/impossible to program more advanced techniques
• deprecated for a while (since OpenGL 3)
• not available in WebGL
– But must do more work on setting up the program
• get shader text (typically from file) • compile shader text
• link shaders into a program
Jochen Lang, EECS jlang@uOttawa.ca
Setting Up for Using a Shader
• Originally different shading languages were used: – ARB assembler, HLSL, GLSL etc.
• Now GLSL is tightly integrated with GL – High level, C like language
– Not vendor specific
– Requires at least OpenGL 2.0
– Used by OpenGL 3.3, OpenGL ES and WebGL
Jochen Lang, EECS jlang@uOttawa.ca
Using a Shader
– Load shader form file or hardcoded string
– Compile shader inside your host code
– Bind shaders into a program
– Typically, have at least:
• Vertex shader (here, triangle.vs)
• Fragment shader (here, triangle.fs)
– Set up buffers and uniforms to transfer data to the shaders
– Use them in the drawing program • see gl.useProgram
Jochen Lang, EECS jlang@uOttawa.ca
A Quick Look at the Vertex Shader
• Vertex shader
#version 300 es
uniform mat4 mvp_matrix;
layout (location=0) in vec4 a_vertex;
layout (location=1) in vec3 a_color;
out vec3 v_color;
void main() {
// map the vertex position into normalized device coordinates gl_Position = mvp_matrix * a_vertex;
// Pass on the color to the fragment shader
v_color = a_color;
}
Jochen Lang, EECS jlang@uOttawa.ca
A Quick Look at the Fragment Shader
• Fragment shader
#version 330 core
#ifdef GL_ES
precision mediump float;
#endif
in vec3 v_color;
out vec4 frag_color;
void main() {
// Pass on the fragment color to the frame buffer frag_color = vec4( v_color, 1.0 );
}
Jochen Lang, EECS jlang@uOttawa.ca
Geometry Transfer
– Objects consist of many vertices, (and normals, texture coordinates, vertex colors)
• Function calls are slow
• Small data size transfer from PC memory to graphics card (GPU) memory has large overhead
• Memory transfer from PC to GPU is slower than memory access on the GPU
– Need method with less function call overhead than immediate mode rendering which can store data on the GPU
No. Vertices : 863,210
Source: Aim@Shape
© VCG-ISTI, P. Cignoni
Jochen Lang, EECS jlang@uOttawa.ca
Buffer Objects
– Buffer objects hold data in OpenGL • also called data stores
– Buffer objects can hold different type of data
• we look first at vertex data
• A buffer object to hold vertex data is also called a Vertex Buffer Object (VBO)
– OpenGL will store the data in PC memory or the GPU to get the best performance
• we can supply hints based on how we intend to use the data
Jochen Lang, EECS jlang@uOttawa.ca
Buffer Memory Allocation
• Need to perform the following steps
1. ask WebGL for a buffer (get an unsigned integer number)
2. bind the buffer to a binding point
• here use the gl.ARRAY_BUFFER binding point
3. Fill the buffer with data
var vertexColorBuffer = gl.createBuffer();
// Bind the buffer object to target gl.bindBuffer(gl.ARRAY_BUFFER, vertexColorBuffer);
// Write date into the buffer object gl.bufferData(gl.ARRAY_BUFFER, vertices, gl.STATIC_DRAW);
Jochen Lang, EECS jlang@uOttawa.ca
Vertex Array Objects (VAO)
– Organize one or several vertex array buffers into a common object
– Available in WebGL 2.0, OpenGL ES 3.0
– OpenGL 4 always use a VAO
– Once the VAO is bound VBOs can be bound to it
// Generate an VAO and bind it
VAO = gl.createVertexArray();
gl.bindVertexArray(VAO);
// Disable it
gl.bindVertexArray(null);
Jochen Lang, EECS jlang@uOttawa.ca
Drawing with a VBO/VAO
– Let OpenGL know how the data is to be used
• index, size of a single data, data type, normalize, stride (if there are gaps in the array), offset
– Enable the attribute in the buffer object for drawing
gl.vertexAttribPointer(a_vertex, 2, gl.FLOAT, false, 5*fsize, 0);
gl.enableVertexAttribArray(a_vertex);
– Drawing with the currently bound VAO
• type of primitive, starting index and number of vertices
gl.drawArrays(gl.TRIANGLES, 0, n);
Jochen Lang, EECS jlang@uOttawa.ca
Program Structure with VAO/VBO
• Set-up in initialization routine
– generate and bind VAO
– generate, bind and fill (multiple) VBO for storing vertices and their attributes (e.g., colors, normals, texture coordinates)
– unbind VAO and possibly go the next set-up
• Drawing routine
– bind the desired VAO
– call draw using data stored in the VBOs of the VAO – unbind the VAO
• Clean-up
– If there is lot of data in WebGL, free up the space taken up by the VAO/VBO (see glDeleteVertexArrays)
Jochen Lang, EECS jlang@uOttawa.ca
Next Lecture
• 2D Transformation
– Textbook: Chapter 6.1, 5.2-5.3
– 2D Transformation
– Translations and homogenous transforms
– Matrix composition and decomposition
– Self-Study: Basic linear algebra, Textbook, Chapter 5.1-5.4
Jochen Lang, EECS jlang@uOttawa.ca
Next lecture
• Drawing and Animation
– Drawing and graphics pipeline – Modelling a Tetrahedron
– Projections from 3D to 2D
– Rasterization
– Animations
Jochen Lang, EECS jlang@uOttawa.ca