Readings on Physically Based Rendering

Over the past two years I’ve done quite a bit of reading on Physically Based Rendering (PBR) and I have collected a lot of references and links which I’ve always had in the back of my mind to share through this blog but never got around doing it. Christmas holidays is probably the best chance I’ll have so I might as well do it now. The list is by no means exhaustive, if you think that I have missed any important references please add them with a comment and I will update it.

Linear Lighting and Shading

There is probably no point in talking about PBR, without first understanding why we should do all lighting and shading in linear space. This is why:

• The Importance of being Linear. A good introduction to the topic.
• Adventures with Gamma-Correct Rendering. Another great introduction by Naty Hoffman
• Gamma FAQ – Frequently Asked Questions about Gamma. Poyton’s FAQ explains a lot of the terminology involved in gamma and linear spaces. While you are there you might as well read the Color FAQ also.
• Uncharted 2: HDR lighting by John Hable.  This is covers many areas, but it has a great introduction to linear lighting+shading. There is also a writeup of this presentation on John’s blog.
• The value of Gamma-compression. Gamma space is not all bad, this is how our eyes perceive light intensity as well as it is the preferred space for image compression and 8-bit image storing.
• Nutty software have a nice WebGL demo of Gamma correction.

Physically based reflection models

Once we are convinced of the importance of linear lighting and shading we can move onto physically based reflection models.

Siggraph’s Physically Based Shading courses provide both an introduction (mainly by Naty Hoffman which is a must read) and in depth coverage of many PBR topics.

• 2017 Course – Presentations from Unity, Infinity Ward, DreamWorks, Framestore, ImageWorks, Pixar.
• 2016 Course – Presentations from DICE, Unreal, Unity, Pixar, ILM. This year there was also a video recording of some of the presentations.
• 2015 Course – Presentations from Activision, Ready at Dawn, Weta Digital, Disney, SledgeHammer. Also Naty Hoffman’s seminal “Introduction to Physically Based Shading in Theory and Practice”presentation video recording is available online.
• 2014 Course – Presentations from Frostbite, Pixar, Tri-ace among others. Frostbite’s PBR presentation course notes (must read) are also available.
• 2013 Course – Presentations from COD:Black Ops II, Unreal Engine 4, The Order:1886 and Pixar among others
• 2012 Course – Presentations from Far Cry 3, TriAce, Disney and Pixar among others
• 2010 Course – Presentations from TriAce, ILM and Sony
• 2006 Course – Very interesting introduction to PBR, including combining PBR and Image based lighting.
• Physically based lighting in COD:Black Ops. There was no PBS course in 2011 but this is a very interesting talk on the topic.
• Peter Shirley’s “Basics of physically based rendering” Siggraph Asia 2012 course notes

Other interesting talks from conferences include

• Michael Drobot’s Lighting of Killzone:Shadowfall  presentation is well worth reading, it covers a lot of PBR related topics.
• Same with the video of GDC 2013 Panel – Metal Gear Solid 5 & Fox Engine.
• Static Sky Unite 2013 Presentation which describes the approach the game team followed to implement Physically Plausible Rendering on mobile platforms, baking a simplified Cook-Torrance BRDF into a lookup table and approximating glossiness using the mip levels of the texture.
• Mastering DX11 with Unity discusses Physically based shaders implemented in Unity
• Physically based shading in Unity 5 GDC2014 presentation by Aras Pranckevičius (and notes).
• “The Order:1886” GDC2014 presentation by David Neubelt and Matt Pettineo is also full of useful information on the game’s material and lighting pipeline.
• Moving to the Next Generation – The Rendering Technology of Ryse GDC2014 presentation by Nicolas Schulz describes among others the shading model used in the game.
• Mastering Physically Based Shading in Unity 5 from Unite2014 (video) describes in depth how Unity handles PBR, good background reading as well.

TriAce’s Research department is doing some excellent work on PBR, unfortunately many of the presentations are in Japanese only. Of special note are the “Practical Physically Based Rendering in Real-Time” talk from GDC 2012 and the “How to Design Your Art Assets for Physically Based Rendering” talk from CEDEC 2012. The latter is in Japanese unfortunately, so if yours is a bit rusty then here is a translation.

The freely available “Programming Vertex  Geometry and Pixel Shaders” e-book provides a great introduction to PBR presenting many BRDFs with shader samples.

Then there is a wealth of information available through numerous blog posts:

• Sébastien Lagarde has written some very informative blog posts on the topic, well worth reading: Adopting a physically based shading model, Feeding a physically based shading model as well as the DONTNOD specular and glossiness chart. Also environment mapping (aka image based lighting) is crucial for realistic looking materials, especially metals, and this post explains how to prepare them with AMD Cubemapgen. A good writeup of the topics described in the blog posts can be found in this fxguide feature article on Remember Me.
• More good introductions to PBR by Julien Guertault and Rory Driscoll
• A good introduction to microfacet BRDFs from Simon’s Tech Blog
• Energy conservation is an important aspect of a BRDF, a great introduction from Rory Driscoll again. Energy conservation is worth pursuing even if you use non realistic lighting models like wrapped-diffuse. On the topic of normalisation, Fabian Giesen shows how the normalisation factor for Phong specular is derived and  The Blinn-Phong Normalization Zoo discusses the various options for the Phong and Blinn-Phong specular.
• Brian Karis posted a great BRDF reference with many options for the various terms. Also worth reading is his Siggraph 2013 presentation on Physically Based Shading in Unreal Engine 4.
• Matt Pettineo’s follow up to the very interesting The Order: 1886 Siggraph 2013 presentation expands on some PBR topics and adds pointers to source code.
• Basic Theory of Physically-Based Rendering by Jeff Russell, another good, maths free, introduction to PBR.
• John Hable has started blogging again, his Filmic Worlds blog has many worth reading posts on PBR.
• A very useful wiki page with lots of information on BRDFs.
• An easy to follow derivation of PI in NDFs by Joshua Barczak
• Steve Anichini describes (among others) PBR in Bioshock Infinite while Spencer Luebbert describes the system from a tech artist’s point of view.
• Physically Based Deferred Rendering in Costume Quest 2, discusses the non-photorealistic approach to PBR taken for this game.
• Physically based shading on mobile, follows Brian Karis’ Siggraph 2013 presentation up discussing adjusting the Unreal 4’s PBR model from mobile platforms.
• Nathan Reed has written 2 very informative posts about Photometry and Radiometry, useful to understand light measurement units: Radiometry Versus Photometry, The Buttered-Toast Model Of Radiometry.
• In Moving Frostbite to PBR Sébastien Lagarde and Charles de Rousiers discuss realistic camera implementation in the context of PBR.
• This series of posts Implementing a Physically Based Camera: Understanding Exposure and Manual Exposure is although worth reading.

If you don’t mind reading academic papers, some links to the original publications for various BRDFs

• Blinn-Phong
• Ashikhmin-Shirley
• Walter (GGX)
• Cook-Torrance
• Torrance-Sparrow
• Ward
• Oren-Nayar
• An overview of BRDF models
• A Survey of Shading Models for Real-time Rendering, another survey of popular BRDF models
• Making shaders more physically plausible, early paper on PBR, touches many PBR related topics
• Morgan McGuire et al describe a nice method to achieve “Physically based” cubemap specular reflections without actually prefiltering the cubemaps with good results.
• “Fast Filtering of Reflection Probes“describes a filtering technique with isotropic kernels that can be used to approximate GGX filtering (among others).
• Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs by Eric Heitz, with an updated version here.
• Filament’s PBR documentation is pretty extensive.

PBR Tools

• BRDF Explorer by Disney Animation is a very useful tool for visualising a large number of BRDFs as well as creating your own.
• cmftStudio is another useful tool for viewing PBR models. It also comes with a cubemap filtering tool.
• Seb Lagarde has modified AMD Cubemapgen to create preconvolved environment maps for PBR.
• Image based lighting baker is another a useful tool for baking diffuse irradiance and specular pre-convolved environment maps.
• There is a free Photoshop plugin for PBR material painting created by Andrew Maximov here. He also offers some PBR textures through his website.

PBR for Artists

The maths of PBR is only half the story and mainly concern the graphics programmers. The other half, of more interest to artists, is how to author the texture assets. In contrast to the first days of PBR, there are now a few great presentations that focus on PBR texture authoring.

• Physically based shading in Real time rendering by Chris­t­ian Schüler, a very easy to follow introduction to PBR without the maths. No texture authoring in this one but well worth a read by artists.
• The Art and Rendering of Remember Me from GDCEurope 2013 by Sébastien Lagarde and Lauren Harduin cover a lot of aspects of PBR and provide artist guidelines for texture authoring.
• Calibrating Lighting and Materials in Far Cry 3 by Stephen McAuley from Siggraph 2012, including capturing and colour correcting textures.
• How to Design Your Art Assets for Physically Based Rendering by Yoshiharu Gotanda, another good set of guidelines on how to author maps for PBR (translation)
• Shining the Light on Crysis 3 by Pierre-Yves Donzallaz provides another good non-maths introduction to PBR, describes Crysis 3’s lighting pipeline and provides guidelines for environmental art.
• RYSE – The transition to Physically Based Shading explains why the team moved to PBS, describes the art pipeline changes provides a high level overview of the game’s lighting system. The tech of Crytek’s Ryse: Son of Rome alsohas some info on the game’s PBS.
• Defrost Game’s Feeding a Physically Based Lighting model is another good math free introduction to PBR.
• A great “PBR for artists by an artist” video presentation by Andrew Maximov.
• A good, maths free and artist friendly, introduction to PBR by Marmoset. While you are there, it is worth reading their Theory of Physically Based Rendering  and PBR Texture Conversion articles as well.
• An easy to read and comprehend introduction to PBR for artists by John Hable.
• The Physically Based Rendering Encyclopedia is a good, easy to understand, summary of many PBR related topics.
• Wolfire’s intro to Physically Based Rendering.

You can also find a modular implementation of the BlinnPhong BRDF model with demonstration of the impact of each term and sample source code in this blog post (shameless plug).

Physically based lights

Physically based shading focuses on how a material responds to light that bounces off the surface in a plausible way. To get better results one should consider the type of light that shines upon the surface as well. Point lights, typically used in games, do not have a counterpart in the real world so several attempts have been made recently to model more realistic, area, lights in games.

• An early attempt to model area lights analytically by John Snyder (Microsoft Research).
• The Art and Rendering of Remember Me by Lagarde and Harduin talks about how they added area lights to the game
• Same with Physically Based Shading in Unreal Engine 4 by Karis, a WebGL demo based on the techniques presented here.
• Killzone:Shadowfall has also put area lights to good use with great results
• A very interesting introduction to Area Light Shadows by Sam Martin
• Some very nice research on Area Lights by Eric Heitz et al.

BRDFs in deferred rendering environments

A variety of BRDFs can easily be implemented in forward rendering architectures. Things become harder with deferred shading though due to the typically low amount of information we can store in a g-buffer. Anisotropic BRDFs are even harder to support, due to the need for a tangent vector. In many cases a single BRDF is enough to represent a variety of materials if you have the capacity to store glossiness and specular colour in the g-buffer. There are cases you might need more though. Some pointers to how people have address this problem so far:

• Bake BRDFs in a 3D lookup table, each layer representing a different BRDF and within each layer addressed by (N.L, N.H). This method was used in STALKER’s deferred shading engine.
• Another approach is to store Material IDs in the g-buffer and during the lighting pass branch to select the desired BRDF. This is the method used in Battlefield 3.
• At Creative Assembly they use the stencil buffer to store the material ID and render each BRDF in a different pass.
• And of course you can always forward render special-case BRDF materials. This is the approach followed recently by Ryse: Son of Rome to render specialised BRDFs.
• The “Deferred Lighting in Uncharted 4” Siggraph 2016 presentation describes how to support multiple BRDFs in a deferred rendering engine with bit-packing in the GBuffer and a table of material-specialised shaders running per-tile.

Not directly related to deferred rendering, being more of a material authoring pipeline, Material Layering is receiving a lot of attention lately after Disney, Unreal Engine 4 and The Order:1886 successfully demonstrated the variety and complexity of materials that can achieve. In short, with this technique we bake parameters of a specific BRDF (that express different materials) into textures which can then be blended before the lighting pass either offline or in the shader. In The Order:1886 blending of different BRDFs is also supported albeit at a greater cost.

Importance Sampling

You will hear Importance Sampling being mentioned in the context of PBR, image based lighting and area lights, quite a lot so it is worth having an idea what it is all about. In short it is a method of sampling a function, image, cubemap etc with a set number of samples by assigning larger weight to important areas of the sampled function (or image, or cubemap). This way we can achieve better representation of the signal without actually increasing the number of samples. This method was feasible only in offline rendering but GPUs are catching up.

• GPU-based Importance Sampling, a method of sampling environment maps for image-based lighting using BRDFs. Some more information about this work here. The original page is not there anymore, you can try your luck with web archive.
• Killzone: Shadowfall uses importance sampling to sample area lights
• If you have the ShaderX7:Advanced Rendering Techniques book it is worth reading the “Efficient post processing with Importance Sampling” chapter by Toth, Szirmay-Kalos and Umenhoffer

Tonemapping

Not directly related to PBR, but when normalising a BRDF the specular highlight intensity can easily reach values above 1 and appear to burnout. For that reason we typically combine PBR with a tonemapping solution. A few pointers to get you started:

• A Closer Look At Tone Mapping, Matt Pettineo provides a showcase of  popular tonemapping operators with sample code.
• Hable’s Uncharted 2 Lighting presentation as well as his blog provide a lot of valuable background info on tonemapping.
• Angelo Pesce’s HDR Workflows is another good introduction to the subject providing many useful insights
• Postprocessing in the Orange box by Alex Vlachos details Valve’s HDR pipeline
• The Programming Vertex, Geometry and Pixel shaders ebook has a nice introduction to HDR/Tonemapping as well.
• A nice WebGL demo showcasing the Reinhard tonemapping operator (original paper).
• This paper which introduces the colour pipelines behind modern feature-film visual-effects and animation is also relevant.

Shader Antialiasing

Again, this topic is not directly related to PBR, but it is a shame to make all that effort to create realistic materials only to have specular highlights crawl and shimmer as the camera moves and surfaces look flatter at a distance.

• Togsvig presented a cheap and nice looking method to adjust the specular power based on the mipmapped normal variation. Stephen Hill has created a WebGL demo of this technique.
• Stephen Hill’s Rock Solid Shading Siggraph talk as well as the related blog post on Specular AA are a must read.
• Spectacular Specular-LEAN and CLEAN specular highlights by Dan Baker details the Specular AA technique used in Civilisation 5 as well as describing the specular aliasing problem.
• Frequency Domain Normal Map Filtering, the specular AA method used in The Order:1886 is based on, calculates an NDF for each texel of the normal map using all normals from the highest-resolution mip level that contribute to a single lower-resolution texel. This NDF is then convolved with the BRDF. The produced BRDF that properly accounts for the variance of all normal map texels for a specific pixel.
• In this blog post Matt published a sample application that implements and showcases popular specular AA techniques including Frequency Domain Normal Map Filtering, well worth a look.
• Antialiasing Physically Based Shading with LEADR Mapping Siggraph 2014 presentation also has lots of good information on antialiasing in PBR pipelines.
• The recent work of Anton Kaplanyan et al on rendering stable subpixel glints and details is also worth following: Real-time Rendering of Procedural Multiscale Materials, Rendering Highly Specular Materials (video).
• Temporal antialiasing is very popular lately, check Marco Salvi’s GDC2016 presentation for an introduction.

I focused on materials readily available on the Internet. There are books worth looking at if you want to find more info though, such as:

• Physically Based Rendering by Matt Pharr and Greg Humphreys, with source code
• Real-Time Rendering, by Tomas Akenine-Möller, Eric Haines, and Naty Hoffman
• High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting 2nd edition by Erik Reinhard, Wolfgang Heidrich, Paul Debevec, Sumanta Pattanaik, Greg Ward and Karol Myszkowski
• “An Efficient and Physically Plausible Real Time Shading Model” by Christian Schüler in ShaderX7 – Advanced Rendering Techniques

As I’ve already mentioned, this list is not exhaustive, if you think that I have missed an important link or topic please add it to the comments sections.

Enjoy!

Edit 03/08/2018 – Added Filament’s PBR documentation reference

Edit 22/11/2017 – Added Siggraph 2017 PBS course link

Edit 11/08/2016 – Added Deferred Lighting in Uncharted 4, Subpixel glint/details rendering links and the video recording of Intro to PBS talk kindly suggested by Naty Hoffman.

Edit 10/08/2016 – Added Siggraph 2016 PBS course link, Area lights with Linearly Transformed Cosines link

Edit 22/10/2015 – Added Wolfire’s intro to Physically Based Rendering

Edit 16/08/2015 – Added Siggraph 2015 PBS course link 

Edit 21/02/2015 – Added Allegorithmic’s PBR guides and Tri-ace’s Siggraph 2014 talk. 

Edit 22/12/2014 – Fixed HDR Workflows broken link and added Cinematic Colour paper suggested by Kyle Hayward

Edit 30/11/2014 – Added Physically based camera links kindly suggested by Seb Lagarde, PBR in Unity kindly suggested by Aras Pranckevičius, Antialasing PBR with LEADR mapping SIG2014 presentation and Marmoset’s Preparing textures for PBR article.

Edit 23/11/2014 – Added Siggraph 2014 Course links, Unreal 4 PBR on mobile, Nathan Reed’s Photometry and Radiometry blog posts and PBR for Costume Quest 2 article. Also added new section about Tools for PBR.

Edit 27/07/2014 – Added Bioshock Infinite PBR posts, PBR Encyclopedia, and Photoshop PBR Plugin.

Edit 11/04/2014 – Replaced broken WebGL gamma correction demo link, thanks to Peter Liu for the heads up.

Edit 27/03/2014 – Added Crytek’s GDC2014 presentation, “The tech of Crytek’s Ryse: Son of Rome” suggested by Sébastien Lagarde and the BRDF wiki page suggested by Peter Liu.

Edit 23/03/2014 – Added “Physically based shading in Unity5”, “The Order: 1886” GDC2014 talks as well as Hable’s Filmic Worlds blog links.

Edit 25/02/2014 – Added “Introduction to PBR for artists” article by John Hable to the PBR for Artists section.

Edit 23/02/2014 – Added “Introduction to PBR” article by Marmoset to the PBR for Artists section.

Edit 11/01/2014 – Added “Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs” technical report as well as Russell’s “Basic Theory of Physically-Based Rendering” article.

Edit 31/01/2014 – Added PBR for Artists video by Andrew Maximov

Edit 03/01/2014 – Added  “An Efficient and Physically Plausible Real Time Shading Model” ShaderX7 book chapter reference kindly suggested by Sébastien Lagarde

Edit 31/12/2013 – Added a few missing links kindly suggested by Sébastien Lagarde and Aras Pranckevičius