Computer Graphics
COMP3421/9415 2021 Term 3 Lecture 15
What did we learn last week?
Advanced Lighting and Maps
¡ñ Multiple addons/corrections for ¡ñ Maps
¡ñ Diffuse, Specular and Normal Maps
¡ñ Extra surface details means less geometry needed
What are we covering today?
Image Reflections
¡ñ A step beyond specular lighting
¡ñ An actual image instead of just the light source
¡ñ Rendering objects with a material like chrome
¡ñ Cube Maps, Skyboxes
¡ñ Environment Mapping
¡ñ Realtime Reflections
First Reflections in Games?
3D’s Bathroom Mirror
There’s a bathroom mirror in a game in 1996
¡ñ Some context . . .
¡ñ Phong shading appears in games in 1999
¡ñ Quake is also released in 1996, with a very
limited lighting model
¡ñ How did this game manage to handle
reflections?
Image credit: 3DRealms and Gearbox Software
What’s required for a believable reflection?
What are we expecting from a virtual mirror?
¡ñ Inverted scene
¡ñ Changes angle of view in realtime
¡ñ Perspective shift based on distance to (and through) mirror
¡ñ Character(s) moving relative to their current position
1996
¡ñ We don’t even have a complete lighting model yet!
¡ñ How was this achieved?
Keep Tricking Those Humans
Never assume technology when dirty tricks will work
¡ñ There is no complicated inverted rendering
¡ñ There’s no calculated perspective
¡ñ No ray tracing of vision “through” the mirror
Trick the humans!
¡ñ There’s a whole other room on the other side
¡ñ It’s 3D so its perspective is automatic
¡ñ There’s a replica of the player that copies your movements
The tricks revealed
If you want to look into it
¡ñ Turning off clipping in the game allows you to walk through walls
¡ñ Turns out, walking “through” a mirror just places you in an inverted scene ¡ñ https://youtu.be/1NG1cIjv-mA?t=33
Is this how it’s done?
No, we’re going to reflect images without tricks
¡ñ It’s interesting to look at though
¡ñ Genuine reflection took a few more years to appear in games
¡ñ But the techniques were already decades old!
¡ñ “Flight of the Navigator” (1986)
Image credit: Disney
Rendering Reflections
Reflections in Polygon Rendering
What’s the basic idea?
¡ñ We could do !
¡ð (the expensive answer to all graphics questions)
¡ñ Cast a ray from a viewer to a reflective object
¡ñ Reflect using surface normals
¡ñ Take colour from where it hits and mix it with the surface
¡ñ This makes sense!
for Reflections
¡ñ A chrome sphere in a scene
¡ñ Some ideas of reflected vision rays
Analysis of Reflections
Pros
¡ñ 100% accurate in theory
¡ñ Deals with any kind of curved or warped reflective surface (including
normal mapped etc)
¡ñ Can sample the colour of the reflecting surface as well as the final point
Cons
¡ñ How expensive is it to find out what object you hit with the reflected ray?
¡ð The answer is the current retail price of a RTX graphics card
¡ð The real answer is close to O(n) where n is the number of triangles in the scene!
A Compromise
Can we get the pros without the cons?
¡ñ The ray bouncing off the shiny surface feels right
¡ñ The collision detection afterwards doesn’t
¡ñ Can we prepare the scene so we’re not doing collision detection per ray
every frame?
Cube Maps
What is a Cube Map?
A particular form of texture
¡ñ Acts like a cube in world space
¡ñ It’s like we’re in the centre of a box looking at
its inner surfaces
¡ñ Colour data is on the 6 faces of the cube
facing inwards
¡ñ We can sample the texture using a direction
vector
Image credit: learnopengl.com
Skybox
A very common use of a cube map
¡ñ Represents everything in the distance
¡ñ Things that can be seen, but don’t need geometry
¡ñ Too far away to need real detail
¡ñ Not just used for reflections!
¡ñ These will usually be our scene backgrounds
Image credit: learnopengl.com
Sampling a Cube Map
We sample a cube map with a direction vector
¡ñ Assumes the cube map is very distant ¡ð Like a directional light
¡ñ Place the centre of the cube at (0,0,0)
¡ñ Interpolate the direction vector until it reaches the cube’s surface
¡ñ Sample the colour from that position
Creation of Cube Maps
The example looks like a photo
¡ñ Photography techniques with chrome spheres
¡ñ Or multiple photos
¡ñ A cube map is a view of a scene looking in
every direction from a single point
Image credit: (c/o https://www.pauldebevec.com/)
Break Time
What’s the most important thing in Graphics?
¡ñ Does the human observer believe it?
¡ñ Is it realistic enough for them?
¡ñ Or are they engaged with the art style?
¡ñ This is Need 4 Speed: Most Wanted (2012)
¡ñ Note how high quality the car is
¡ñ vs how plain and repetitive the environment
is
¡ñ The effort is where the human’s eyes are!
Image credit: EA
Environment Mapping
A Cube Map of the Environment
If we have a cube map
¡ñ Represents the environment around an object
¡ñ Can be sampled with a direction vector
¡ñ We can now reflect vectors off shiny objects
¡ð We’ll use information per fragment
¡ð Direction to the Viewer and Surface Normal
¡ñ And see where they reflect the environment
In OpenGL
OpenGL has “GL_TEXTURE_CUBE_MAP”
¡ñ A convenient set of 6 textures
¡ñ We’ll still have to generate 6 individual textures and bind them to the
cube map
¡ñ Sampling from the cube map is done with a direction vector
¡ñ OpenGL handles sampling the actual colour
Static Environment Reflections
Pre-prepared vs dynamically created
¡ñ Environment Mapping gives us the static background of the scene
¡ñ For a lot of reflections this is enough
¡ñ But we want foreground objects and
movement to also be reflected
¡ñ Can we create a cubemap dynamically?
¡ñ We’re going to need some more tech to do this
in realtime . . .
Metal Mario uses Environment Mapping in Mario 64 (1996) Image credit: Nintendo
Frame Buffers
Framebuffer Revision
A target to render to
¡ñ The default framebuffer is the last buffer we write colours to
¡ñ Its RGB values are sent to the monitor
¡ñ We can create other framebuffers
¡ñ We can also render to those framebuffers
¡ñ They can store textures (colour) and other data
¡ñ In HDR, we created a framebuffer the same size as our monitor/window
¡ñ But we’re not limited to that!
Framebuffers and Cube Maps
We can use the same concept!
¡ñ Cube maps are textures
¡ñ Frame buffers can store textures
¡ñ Frame buffers can be written to in realtime
¡ñ We can write to the 6 faces of the cube map while rendering our frame!
Rendering to Textures
Render to Texture
A technique to create texture data in realtime
¡ñ Previously our textures were images given to our application
¡ñ But a texture in OpenGL is a colour data buffer
¡ñ Buffers can be written to by our shaders
¡ñ eg: The Framebuffer colour is set by the final fragcolor in our shaders
¡ñ So our 6 framebuffers that make up our cube map could be dynamically
created
Render Targets and Cube Maps
Working with Framebuffers
¡ñ By default we’re rendering to the main framebuffer
¡ñ But at any time in our fragment shader, we can bind to a different
framebuffer
¡ñ So we can actually render 6 different times
¡ñ Each time rendering to a face of the cube map
Realtime Reflections
Making a cube map dynamically
A few renders in one
¡ñ Set up a temporary camera where your shiny object is
¡ñ Set up a cube map with its 6 frame buffers
¡ñ Render 6 times, aiming the temporary camera in each of 6 directions
¡ð Positive and negative in each axis
¡ð Each render writes data to one face of the cube map
¡ñ Then use that cube map to sample from for reflections
Realtime Cube Map Creation
¡ñ Remove the shiny object and replace it with the new camera
¡ñ Take 6 renders in different directions and write the results to the cube
map’s framebuffer textures
Realtime Reflections
With a new Cube Map each frame
¡ñ This cube map is created at the location of the reflective object
¡ñ Reflections from the main camera’s viewpoint can use that cube map
¡ñ Reflect the viewer direction vector using the surface normal for each
fragment
Realtime Cube Maps
Pros
¡ñ Able to reflect dynamic objects
¡ñ Able to reflect more than just the Environment Map
Cons
¡ñ Wait, 7 renders per frame?
¡ñ Faster than ray tracing, but still not “fast”
¡ñ We have tricks to save processing time . . .
Shortcuts for efficiency
How do we reduce the cost of the 7 renders per frame?
¡ñ Leverage the fact that reflections are usually distorted and unclear
¡ñ Cube map is not created every frame!
¡ð The scene around the shiny object might not change that much
¡ð Only render the cube map every few frames
¡ñ Lower the resolution of the cube map
¡ð Reflections don’t need the same resolution as the screen
¡ð The cube map can be very low resolution and still get close to one reflected texel per pixel
¡ñ Leave out the downwards render
¡ð The Need 4 Speed optimisation
¡ð If very few of the reflections are going to use the downward face, we won’t need to render
it
What did we learn today?
Reflecting on Reflections
¡ñ Ideas about how to handle reflections
¡ñ Cube Maps
¡ð Sampling using direction vectors
¡ñ Environment Mapping
¡ñ Frame Buffers and Rendering to Textures
¡ñ Realtime Cube Map creation