Using the DirectXShaderCompiler C++ API - Revised

24 Mar 2020 - Simon Coenen - Reading time: 6 mins

Shortly after my previous post, DirectX Developer Day has happened and the DirectX team has shared a great amount of new updates and insights to the world including new additions to the DirectX Shader Compiler. The main thread is that the API has become a lot easier to use and is streamlined so that both the C++ and CLI interfaces share the same code path. This makes it a lot easier for developers to work with. So now things have changed quite a bit.

The main thing that got updated are the introduction of IDxcCompiler3 which is has the new interfaces that are streamlined with the CLI. Secondly, it’s now a lot clearer what “parts” the compiler outputs and it’s more easy to isolate certain parts like reflection and PDBs and possibly strip them to be processed later.

I highly recommend watching all the videos on the DirectX’s YouTube channel!

Getting Started

At the time of writing this, there is not yet an official release with the new DXC updates however the latest build binaries can be downloaded from the repo’s AppVeyor.

There are quite a lot of binaries in there but to use the C++ API you only need a few:

  • dxcompiler.lib - The compiler dll link lib
  • dxcompiler.dll - The compiler’s frontend
  • dxcapi.h - The header with the interfaces. You only need this single header. (/include/dxc/dxcapi.h)

Read More...


Using the DirectXShaderCompiler C++ API

04 Mar 2020 - Simon Coenen - Reading time: 17 mins

Shader Model 6 has existed for quite a while now and after reading this great article from Francesco Cifariello Ciardi about scalarizing the light loop in tiled/clustered lighting, I wanted to try out Wave Intrisics. The old fxc.exe compiles to DXBC and only supports up to Shader Model 5.1. Microsoft has since introduced their new llvm-based compiler called DirectXShaderCompiler (DXC) which compiles to DXIL. It’s completely open-source on GitHub here: https://github.com/microsoft/DirectXShaderCompiler It both provides command line tools and a C++ API for compiling, validating and using shaders with SM 6.0 and up.

Fxc has existed for quite a long time now and there are guides for it everywhere however Dxc is quite a bit younger and I didn’t find too much about it. The documentation for it is quite minimal and incomplete which is surprising considering DirectX is generally really well documented. Using Dxc’s commandline tools are quite straight forward and self-explanatory however I’ve always like to compile my shaders at runtime because it eventually require no effort when modifying shaders and having to recompile them. Unfortunately, I didn’t find too much documentation on the C++ API.

I’ve found a few articles here and there about the very basic setup for this however most of the were very minimal and didn’t really cover things like compile-arguments, defines, include handling, validation, reflection, debug-data, … which are quite important to know about.

(20/03/2020): DirectX Developer day has happened and there’s been a super good talk by one of the DXC developers and they gave a great walkthrough of the interface. I’d definitely recommend watching that as things described in this post might not all be correct. Check it out here!

Read More...


Optimizing spotlight intersection in tiled/clustered light culling

17 Aug 2019 - Simon Coenen - Reading time: 12 mins

I’ve been implementing both tiled and clustered light culling with forward rendering recently and one of the things I never was quite happy with is the way spot lights were tested against the frustum voxel/AABBs. I wanted to write up something about my entire culling implementation but then I got a bit sidetracked by looking into way to improve cone culling specifically and I thought the spot light testing is an interesting case on its own.

To give a very brief summary of what light culling actually is:

Light culling is an optimization method to reduce the amount of lights that are considered when shading individual pixels by placing lights in either a 2D (tiled) or 3D (clustered) grid based on their position, dimensions and possibly other parameters. Before the shading pass, a compute shader splits up the view frustum in buckets and loops over all the lights and places them in the buckets that it intersects with. Later during shading, using the pixel position, the appropriate light bucket can be calculated and only the lights in that bucket need to be considered for that pixel. The technique can also be leveraged to accelerate other things like decals, environment probes, … This greatly improves shading of a scene with a lot of dynamic lights. It’s a very common technique used in many games and engines.

Spot light culling can be challenging to find a good intersection method for. There are several different approaches to this and depending on how far you think this through, there are some significant optimizations to be made here compared to naive methods.

I’ve gained lots of insight from other articles which really helped me learn about all the methods. One of which is the amazing article of Bartłomiej Wroński and I really recommend checking it out if you want some more in-depth information about this. The others are mentioned below in case you want to learn more about the subject.

Read More...


Pak files - Virtual file system

10 Jul 2019 - Simon Coenen - Reading time: 16 mins

Packaging assets in large binary blobs in games is quite common. In 1992, Wolfenstein 3D introduced so called “WAD” files (Stands for “Where’s All the Data?”). This file format has been used after on Doom and eventually pretty much all games currently today in some form. It gives more flexibility to users (and developers) to create patches, mods, it provides opportunities for security measures but more importantly, it improves performance significantly as having only a few large binary files to read from is much faster than reading many small files.

Today, many game engines have adopted the idea of having large files of binary asset data. Unreal Engine uses .pak files, Unity uses .assets files, Anvil uses .forge files, .raf in League of Legends, …

As a small project, I’ve decided to look into creating a similar format. On top of what I’ve described above, the files in a pak file are usually also compressed. This can significantly reduce disk size and possibly even improving performance when disk access is slow and the cost to decompress is low.

Read More...


Basic compile-time type information using constexpr

21 Jun 2019 - Simon Coenen - Reading time: 11 mins

When working on my game engine project, I always get distracted by new interesting things or thoughts I want to look into and one of them was reflection. Almost all commercial game engine have some kind of reflection that makes GUI editors and visual scripting possible. Unlike those engines, I didn’t look into full all-the-way reflection because I didn’t want to bloat my code with that and maintain it. I was mostly interested in very simple type reflection so before you start frowning when looking at the code, this is not meant to be a complete reflection system at all! This is quite basic but I found it to be an interesting use of compile-time expressions.

All the code can be found on GitHub. The relevant files to look at are:

What it does

I mainly started working on this out of curiosity and to see how far I could take it without having to modify loads of files everywhere. I can definitely take it much further but in its current state I can retrieve type information and create instances of classes using string hashes and object factories. Getting type information is compile-time without any runtime cost.

I currently use it for getting game object components, object instantiation, shader variable addressing and general dynamic casting.

Read More...


Natvis in Visual Studio

13 Jun 2019 - Simon Coenen - Reading time: 5 mins

When working on the C++ delegates (see previous post), I found out about a thing called “natvis” and I never actually heard of it before even though it’s always been right under my nose when debugging Unreal Engine projects. It’s a really awesome and powerful debugging tool that allows you to define how classes should be visualized in the Visual Studio “Watch” windows and hover windows.

So instead of having to dig through a load of expanders to find what you’re looking for in an object, you can fully customize the window to show only the relevant information of that object. Natvis is a xml-based file that you simply include in your Visual Studio project that Just Works™.

Many of you might already know about this but there’s a small detail that helps you with debugging natvis that you might not know about!

Difference

Read More...