Vulkan support for SHADER_EARLY_DEPTH_TEST and fix to conservative depth optimizations

[dzamkov]

Connections
#3961: Resolves for Vulkan and GL backends
#4408: Resolves for Vulkan backend

Description

1. Support for forced early fragment tests and conservative depth in the Vulkan/SPIR backend, using the Naga-specific @early_depth_test attribute
2. Fixes design issue which prevents useful conservative depth optimizations in GLSL

Originally, I had only planned to do the first, but while reviewing the specs and doing research for this change, I discovered that the current design of Naga's EarlyDepthTest doesn't make sense. There are two use cases for early fragment tests:

1. Forcing early fragment tests while disabling late tests: Typically used when the fragment shader writes to storage buffers and these writes must be prevented when depth/stencil tests fail
2. Permitting early fragment tests while still having late tests enabled: Typically used when the fragment shader changes the frag_depth in a predictable way, allowing an additional early depth/stencil test to optimize away unnecessary shader invocations without changing behavior

These use cases are incompatible with each other. In both Vulkan and GLSL, forcing early fragment tests disables late tests, which means that writing to frag_depth is either useless or an error. Currently, Naga doesn't allow writing a conservative depth specifier without also forcing early fragment tests, invalidating the conservative depth specifier.

To salvage conservative depth optimizations, I have reworked EarlyDepthTest into an enum with variants corresponding to the two use cases above. To make this more explicit, @early_depth_test(force) is now required to force early fragment tests.

Testing

• Added snapshot tests and reviewed output
• Integrated into a personal project and verified that forced early fragment tests work as expected

Squash or Rebase?

Squash

Checklist

• Run cargo fmt.
• Run taplo format.
• Run cargo clippy --tests. If applicable, add:
  • --target wasm32-unknown-unknown
• Run cargo xtask test to run tests.
• If this contains user-facing changes, add a CHANGELOG.md entry.

[Wumpf]

Looking good, thanks for investigating this! Left a few doc improvement suggestions.

(I rarely do Naga reviews, so would be good if someone of the Naga-regulars has another look as well :))

[Wumpf]

this should also document the non-force case and what variants it has

[dzamkov]

Added a link to ConservativeDepth, which further documents the conservative depth variants

[Wumpf]

in the PR description you wrote that specifying both is invalid. But that's not documented here and not validated on parsing

[dzamkov]

This statement is true as written: the commonality between both variants is that it allows early depth tests in scenarios where the WGPU/WGLSL specs normally wouldn't permit it. Forcing an early depth test necessarily implies that an early depth test is allowed.

I think the fact that this is an enum should be sufficient documentation that the two variants are mutually exclusive?

Can you clarify what you mean by validation? It's not possible to specify both variants in WGSL using the early_depth_test attribute, since you have to either specify force or a conservative depth. In the other shading languages, its technically not illegal (as far as I know) to both force early fragment tests and provide a conservative depth specifier, merely pointless.

[Reun85]

In the other shading languages, its technically not illegal (as far as I know) to both force early fragment tests and provide a conservative depth specifier, merely pointless.

Ray marchers benefit immensely from being able to use both the depth_greater specifier and early depth tests, as they are incredibly expensively operations only executable in fragment shaders.

[dzamkov]

For ray marchers, you should only specify depth_greater and hope that the driver decides to perform early depth tests (which it probably will). If you force early depth tests, that will disable depth writes from the shader and give you incorrect results. See:

• OpenGL Spec: 14.9.4:

If early fragment tests are enabled, any depth value computed by the fragment
shader has no effect

• Vulkan Spec:

  Not as explicit, but says EarlyFragmentTests moves fragment shading after the depth test and DepthGreater is only used to allow an additional depth test before fragment shading. Thus, when used in combination, there will be no depth test after shading to write out your FragDepth value

• Metal Spec: 5.1.2

It is an error if the return type of the fragment function declared with the
[[early_fragment_tests]] attribute includes a depth or stencil value; that is, if the return
type of this fragment function includes an element declared with the
[[depth(depth_attribute)]] or [[stencil]] attribute.

• Helpful blog post that goes over the different depth testing modes

[Wumpf]

Since this is unfortunately the only actual wgpu facing piece where one could look up what the wgsl syntax looks like (@jimblandy is that true?), it would be great to again list out the different variants and document what they mean (with links to other resources if seemed fit)

[dzamkov]

Added additional documentation