Using the DirectXShaderCompiler C++ API - Revised

24 Mar 2020 - Simon Coenen - Reading time: 6 mins - Comments

Shortly after my previous post, DirectX Developer Day has happened and the DirectX team has shared a great amount of new updates and insights to the world including new additions to the DirectX Shader Compiler. The main thread is that the API has become a lot easier to use and is streamlined so that both the C++ and CLI interfaces share the same code path. This makes it a lot easier for developers to work with. So now things have changed quite a bit.

The main thing that got updated are the introduction of IDxcCompiler3 which is has the new interfaces that are streamlined with the CLI. Secondly, it’s now a lot clearer what “parts” the compiler outputs and it’s more easy to isolate certain parts like reflection and PDBs and possibly strip them to be processed later.

I highly recommend watching all the videos on the DirectX’s YouTube channel!

Getting Started

At the time of writing this, there is not yet an official release with the new DXC updates however the latest build binaries can be downloaded from the repo’s AppVeyor.

There are quite a lot of binaries in there but to use the C++ API you only need a few:

  • dxcompiler.lib - The compiler dll link lib
  • dxcompiler.dll - The compiler’s frontend
  • dxcapi.h - The header with the interfaces. You only need this single header. (/include/dxc/dxcapi.h)

Compiling

The new update comes with a new interface IDxcUtils, this interface essentially replaces IDxcLibrary. The ‘Utils’ interface provides all the functionality to create data blobs. Besides that, just like before, creating a blob remains the same:

ComPtr<IDxcUtils> pUtils;
DxcCreateInstance(CLSID_DxcUtils, IID_PPV_ARGS(pUtils.GetAddressOf()));
ComPtr<IDxcBlobEncoding> pSource;
pUtils->CreateBlob(pShaderSource, shaderSourceSize, CP_UTF8, pSource.GetAddressOf());

Now introducing IDxcCompiler3, the Compile function no longer provides separate input arguments for defines. This now all needs to get passed through as compile arguments using the -D argument. Note the use of std::vector is just for simplicity’s sake.

std::vector<LPWSTR> arguments;
//-E for the entry point (eg. PSMain)
arguments.push_back(L"-E");
arguments.push_back(entryPoint);

//-T for the target profile (eg. ps_6_2)
arguments.push_back(L"-T");
arguments.push_back(target);

//Strip reflection data and pdbs (see later)
arguments.push_back(L"-Qstrip_debug");
arguments.push_back(L"-Qstrip_reflect");

arguments.push_back(DXC_ARG_WARNINGS_ARE_ERRORS); //-WX
arguments.push_back(DXC_ARG_DEBUG); //-Zi
arguments.push_back(DXC_ARG_PACK_MATRIX_ROW_MAJOR); //-Zp

for (const std::wstring& define : defines)
{
    arguments.push_back(L"-D");
    arguments.push_back(define.c_str());
}

DxcBuffer sourceBuffer;
sourceBuffer.Ptr = pSource->GetBufferPointer();
sourceBuffer.Size = pSource->GetBufferSize();
sourceBuffer.Encoding = 0;

ComPtr<IDxcResult> pCompileResult;
HR(pCompiler->Compile(&sourceBuffer, arguments.data(), (uint32)arguments.size(), nullptr, IID_PPV_ARGS(pCompileResult.GetAddressOf())));

//Error Handling
ComPtr<IDxcBlobUtf8> pErrors;
pCompileResult->GetOutput(DXC_OUT_ERRORS, IID_PPV_ARGS(pErrors.GetAddressOf()), nullptr);
if (pErrors && pErrors->GetStringLength() > 0)
{
    MyLogFunction(Error, (char*)pErrors->GetBufferPointer());
}

Now before you stop reading and being happy that you compile a shader, wait! You’re missing out on the main reason why this update is such a big deal! You might have noticed the two arguments -Qstrip_debug and -Qstrip_reflect.

Stripping parts

I’ve found that previously, stripping debug data or reflection data hasn’t always been doing what I would expect. With the new updates, this becomes a lot easier.

The output of the compiler now is a IDxcResult (as opposed to IDxcOperationResult). It has a method GetOutput which allows you to extract a part of the output. This can be one of the following: (defined in dxcapi.h)

  • DXC_OUT_OBJECT
  • DXC_OUT_ERRORS
  • DXC_OUT_PDB
  • DXC_OUT_SHADER_HASH
  • DXC_OUT_DISASSEMBLY
  • DXC_OUT_HLSL
  • DXC_OUT_TEXT
  • DXC_OUT_REFLECTION
  • DXC_OUT_ROOT_SIGNATURE

In the code example above, I added the two arguments -Qstrip_debug and -Qstrip_reflect. The effect of this is that the compiler will strip both the shader PDBs and reflection data from the Object part. The important thing here is that it will still be in the compile result and can be extracted using DXC_OUT_PDB and DXC_OUT_REFLECTION respectively. The result of the flags are that it will no longer be embedded in the DXC_OUT_OBJECT part which keeps the actual shader object nice and slim. I would definitely advise to always use these flags. Pix also understands separate PDBs for shaders.

So now, it’s a lot more clear and easy to specifically get parts of the shader. Getting shader PDBs is done as follows:

ComPtr<IDxcBlob> pDebugData;
ComPtr<IDxcBlobUtf16> pDebugDataPath;
pCompileResult->GetOutput(DXC_OUT_PDB, IID_PPV_ARGS(pDebugData.GetAddressOf()), pDebugDataPath.GetAddressOf());

The function has one extra argument which can be retrieved as a IDxcBlobUtf16. This contains the path that is baked into the shader object to refer to the part in question. So if you want to save the PDBs to a separate file, use this name so that Pix will know where to find it.

For reflection data: (library reflection is similar but with ID3D12LibraryReflection)

ComPtr<IDxcBlob> pReflectionData;
pCompileResult->GetOutput(DXC_OUT_REFLECTION, IID_PPV_ARGS(pReflectionData.GetAddressOf()), nullptr);
DxcBuffer reflectionBuffer;
reflectionBuffer.Ptr = pReflectionData->GetBufferPointer();
reflectionBuffer.Size = pReflectionData->GetBufferSize();
reflectionBuffer.Encoding = 0;
ComPtr<ID3D12ShaderReflection> pShaderReflection;
pUtils->CreateReflection(&reflectionBuffer, IID_PPV_ARGS(pShaderReflection.GetAddressOf()));

On a last note, as mentioned in the video, none of the DXC interfaces are thread-safe and it is advised to have an instance of each interface for each thread. DxcCreateInstance is an exception and is thread-safe.