Understanding Perspective Projection in 3D Games

Perspective matrices have been used behind the scenes since the inception of 3D gaming, and the majority of vector libraries will have built-in helper functions to construct them for you. But what if you wanted to know the reasoning behind how these matrices are constructed.

I start off with a brief introduction of computer graphics and the key ideas behind 3D rendering. I differentiate between Image Order Rasterization typically used by Ray tracers, and object order rasterization as used in most video games. I then present an animated walkthrough showing the full derivation of both the orthographic and perspective projection matrices.

My derivation here is focused on the Vulkan API, but the reasoning applies just as well to other API's such as OpenGL or direct X, with the only notable differences being the size of the canonical viewing volumes, and the handedness and conventions of the xyz coordinate systems.

0:00 How does 3D graphics work?
2:05 Image versus object order rendering
2:51 The Orthographic Projection matrix
5:17 The perspective transformation
7:08 Homogeneous Coordinate division
8:27 Constructing the perspective matrix
10:29 Non-linear z depths and z fighting
11:30 The perspective projection transformation

** Resources **
Fundamentals of Computer Graphics by Peter Shirley & Steve Marschner
http://www.songho.ca/opengl/gl_projectionmatrix.html
https://matthewwellings.com/blog/the-new-vulkan-coordinate-system/
https://www.insider.com/pixars-animation-evolved-toy-story-2019-6

** Attributions**
Ray tracing graphics cards - Photo by Nana Dua from Pexels
Rubiks cube - Photo by Mathias P.R. Reding from Pexels
Ray tracing in video games - https://www.digitaltrends.com/gaming/battlefield-v-dxr-ray-tracing-tested/
https://en.wikipedia.org/wiki/Line%E2%80%93plane_intersection