Initial work for "global generic value parameters" (shader-slang#1127)

* Initial work for "global generic value parameters"

The main new feature here is support for the `__generic_value_param` keyword, which introduces a *global generic value parameter*.
For example:

    __generic_value_param kOffset : uint = 0;

This declaration introduces a global generic value parameter `kOffset` of type `uint` that has a nominal default value of zero.

The broad strokes of how this feature was added are as follows:

* A new `GlobalGenericValueParamDecl` AST node type is introduces in `slang-decl-defs.h`

* A new `parseGlobalGenericValueParamDecl` subroutine is added to `slang-parser.cpp`, and is added to the list of declaration cases as the callback for the `__generic_value_param` name.

* Cases for `GlobalGenericValueParamDecl` are added to the declaration checking passes in `slang-check-decl.cpp`, mirroring what is done for other variable declaration cases.

* A case for `GlobalGenericValueParamDecl` is aded to the `Module::_collectShaderParams` function, so that it is recognized as a kind of specialization parameter. This introduces a specialization parameter of flavor `SpecializationParam::Flavor::GenericValue` (which was already defined before this change, although it was unused).

* A case for `SpecializationParam::Flavor::GenericValue` is added in `Module::_validateSpecializationArgsImpl` to check that a specialization argument represents a compile-time-constant value (not a type).

* A case for `GlobalGenericValueParmDecl` is introduced in `slang-lower-to-ir.cpp` that introduces a global generic parameter in the IR

* The `IRBuilder` is extended to support creating `IRGlobalGenericParam`s for the distinct cases of type, witness-table, and value parameters. The same IR instruction type/opcode is used for all cases, and only the type of the IR instruction differs.

* The existing mechanisms for lowering specialization arguments to the IR, and doing specialization on the IR itself Just Work with global generic value parameters since they already support value parameters on explicit generic declarations.

That's the santized version of things, but there were also a bunch of cleanups and tweaks required along the way:

* The `SpecializationParam` type was extended to also track a `SourceLoc` to help in diagnostic messages, which meant some churn in the code that collects specialization parameters.

* The `_extractSpecializationArgs` function is tweaked to support any kind of "term" as a specialization argument (either a type or a value).

* To allow *parsing* specialization arguments that can't possibly be types (e.g., integer literals) we replace the existing `parseTypeString` routine with `parseTermString` and then in `parseTermFromSourceFile` call through to a general case of expression parsing (which can also parse types) rather than only parsing types directly.

* Right before doing back-end code generation, we check if the program we are going to emit has remaining (unspecialized) parameters, in which case we emit a diagnostic message for the parameters that haven't been specialized rather than go on to emit code that will fail to compile downstream.

* Within the `render-test` tool we collapse down the arrays that held both "generic" and "existential" specialization arguments, so that we just have *global* and *entry-point* specialization argument lists. This mirrors how Slang has worked internally for a while, but the difference hasn't been important to the test tool because no tests currently mix generic and existential specialization. The logic for parsing `TEST_INPUT` lines has been streamlined down to just the global and entry-point cases, but the pre-existing keywords are still allowed so that I don't have to tweak any test cases.

There are several significant caveats for this feature, which mean that it isn't really ready for users to hammer on just yet:

* There is no support for `Val`s of anything but integers, so there is no way to meaningfully have a generic value param with a type other than `int` or `uint`.

* We allow for a default-value expression on global generic parameters, but do not actually make use of that value for anything (e.g., to allow a programmer to omit specialization arguments), nor check that it meets the constraints of being compile-time constant.

* Global generic value parameters are *not* currently being treated the same as explicit generic parameters in terms of how they can be used for things like array sizes or other things that require constants. This will probably be relaxed at some point, but allowing a global generic to be used to size an array creates questions around layout.

* The IR optimization passes in Slang currently won't eliminate entire blocks of code based on constant values, so using a global generic value parameter to enable/disable features will *not* currently lead to us outputting drastically different HLSL or GLSL. That said, we expect most downstream compilers to be able to handle an `if(0)` well.

* Fix regression for tagged union types

The change that made specialization arguments be parsed as "terms" first, and then coerced to types meant that any special-case logic that is specific to the parsing of types would be bypassed and thus not apply.
Most of that special-case logic isn't wanted for specialization arguments, since it pertains to cases were we want to, e.g, declare a `struct` type while also declaring a variable of that type.
The one special case that *is* useful is the `__TaggedUnion(...)` syntax, which is the only way to introduce a tagged union type right now.

In order to get that case working again, all I had to do was register the existing logic for parsing `__TaggedUnion` as an expression keyword with the right callback, and the existing logic in expression parsing kicks in (that logic was already handling expression keywords like `this` and `true`).

I left in the existing logic for handling `__TaggedUnion` directly where types get parsed, rather than try to unify things.
A better long-term fix is to make the base case for type parsing route into `parseAtomicExpr` so that the two paths share the core logic.
That change should probably come as its own refactoring/cleanup, because it creates the potential for some subtle breakage.

* fixup: typo

Author: Tim Foley

source/slang/slang-check-decl.cpp

@@ -49,6 +49,11 @@ namespace Slang
             checkVarDeclCommon(varDecl);
         }
 
+        void visitGlobalGenericValueParamDecl(GlobalGenericValueParamDecl* decl)
+        {
+            checkVarDeclCommon(decl);
+        }
+
         void visitImportDecl(ImportDecl* decl);
 
         void visitGenericTypeParamDecl(GenericTypeParamDecl* decl);
@@ -114,6 +119,11 @@ namespace Slang
             checkVarDeclCommon(varDecl);
         }
 
+        void visitGlobalGenericValueParamDecl(GlobalGenericValueParamDecl* decl)
+        {
+            checkVarDeclCommon(decl);
+        }
+
         void visitEnumCaseDecl(EnumCaseDecl* decl);
 
         void visitEnumDecl(EnumDecl* decl);

source/slang/slang-check-shader.cpp

@@ -52,7 +52,8 @@ namespace Slang
         /// Recursively walk `type` and add any existential/interface specialization parameters to `ioSpecializationParams`.
     static void _collectExistentialSpecializationParamsRec(
         SpecializationParams&   ioSpecializationParams,
-        Type*                   type)
+        Type*                   type,
+        SourceLoc               loc)
     {
         // Whether or not something is an array does not affect
         // the number of existential slots it introduces.
@@ -66,7 +67,8 @@ namespace Slang
         {
             _collectExistentialSpecializationParamsRec(
                 ioSpecializationParams,
-                parameterGroupType->getElementType());
+                parameterGroupType->getElementType(),
+                loc);
             return;
         }
 
@@ -81,6 +83,7 @@ namespace Slang
                 //
                 SpecializationParam specializationParam;
                 specializationParam.flavor = SpecializationParam::Flavor::ExistentialType;
+                specializationParam.loc = loc;
                 specializationParam.object = type;
                 ioSpecializationParams.add(specializationParam);
             }
@@ -112,7 +115,8 @@ namespace Slang
     {
         _collectExistentialSpecializationParamsRec(
             ioSpecializationParams,
-            GetType(paramDeclRef));
+            GetType(paramDeclRef),
+            paramDeclRef.getLoc());
     }
 
 
@@ -147,13 +151,15 @@ namespace Slang
             {
                 SpecializationParam param;
                 param.flavor = SpecializationParam::Flavor::GenericType;
+                param.loc = genericTypeParam->loc;
                 param.object = genericTypeParam;
                 m_genericSpecializationParams.add(param);
             }
             else if(auto genericValParam = as<GenericValueParamDecl>(m))
             {
                 SpecializationParam param;
                 param.flavor = SpecializationParam::Flavor::GenericValue;
+                param.loc = genericValParam->loc;
                 param.object = genericValParam;
                 m_genericSpecializationParams.add(param);
             }
@@ -1020,9 +1026,21 @@ static bool doesParameterMatch(
                 //
                 SpecializationParam specializationParam;
                 specializationParam.flavor = SpecializationParam::Flavor::GenericType;
+                specializationParam.loc = globalGenericParam->loc;
                 specializationParam.object = globalGenericParam;
                 m_specializationParams.add(specializationParam);
             }
+            else if( auto globalGenericValueParam = as<GlobalGenericValueParamDecl>(globalDecl) )
+            {
+                // A global generic type parameter declaration introduces
+                // a suitable specialization parameter.
+                //
+                SpecializationParam specializationParam;
+                specializationParam.flavor = SpecializationParam::Flavor::GenericValue;
+                specializationParam.loc = globalGenericValueParam->loc;
+                specializationParam.object = globalGenericValueParam;
+                m_specializationParams.add(specializationParam);
+            }
             else if( auto importDecl = as<ImportDecl>(globalDecl) )
             {
                 // An `import` declaration creates a requirement dependency
@@ -1425,16 +1443,20 @@ static bool doesParameterMatch(
             auto& arg = args[ii];
             auto& param = m_specializationParams[ii];
 
-            auto argType = arg.val.as<Type>();
-            SLANG_ASSERT(argType);
-
             switch( param.flavor )
             {
             case SpecializationParam::Flavor::GenericType:
                 {
                     auto genericTypeParamDecl = param.object.as<GlobalGenericParamDecl>();
                     SLANG_ASSERT(genericTypeParamDecl);
 
+                    RefPtr<Type> argType = as<Type>(arg.val);
+                    if(!argType)
+                    {
+                        sink->diagnose(param.loc, Diagnostics::expectedTypeForSpecializationArg, genericTypeParamDecl);
+                        argType = getLinkage()->getSessionImpl()->getErrorType();
+                    }
+
                     // TODO: There is a serious flaw to this checking logic if we ever have cases where
                     // the constraints on one `type_param` can depend on another `type_param`, e.g.:
                     //
@@ -1520,6 +1542,13 @@ static bool doesParameterMatch(
                     auto interfaceType = param.object.as<Type>();
                     SLANG_ASSERT(interfaceType);
 
+                    RefPtr<Type> argType = as<Type>(arg.val);
+                    if(!argType)
+                    {
+                        sink->diagnose(param.loc, Diagnostics::expectedTypeForSpecializationArg, interfaceType);
+                        argType = getLinkage()->getSessionImpl()->getErrorType();
+                    }
+
                     auto witness = visitor.tryGetSubtypeWitness(argType, interfaceType);
                     if (!witness)
                     {
@@ -1539,6 +1568,29 @@ static bool doesParameterMatch(
                 }
                 break;
 
+            case SpecializationParam::Flavor::GenericValue:
+                {
+                    auto paramDecl = param.object.as<GlobalGenericValueParamDecl>();
+                    SLANG_ASSERT(paramDecl);
+
+                    // Now we need to check that the argument `Val` has the
+                    // appropriate type expected by the parameter.
+
+                    RefPtr<IntVal> intVal = as<IntVal>(arg.val);
+                    if(!intVal)
+                    {
+                        sink->diagnose(param.loc, Diagnostics::expectedValueOfTypeForSpecializationArg, paramDecl->getType(), paramDecl);
+                        intVal = new ConstantIntVal(0);
+                    }
+
+                    ModuleSpecializationInfo::GenericArgInfo expandedArg;
+                    expandedArg.paramDecl = paramDecl;
+                    expandedArg.argVal = intVal;
+
+                    specializationInfo->genericArgs.add(expandedArg);
+                }
+                break;
+
             default:
                 SLANG_UNEXPECTED("unhandled specialization parameter flavor");
             }
@@ -1556,27 +1608,17 @@ static bool doesParameterMatch(
     {
         auto linkage = componentType->getLinkage();
 
+        SharedSemanticsContext semanticsContext(linkage, sink);
+        SemanticsVisitor semanticsVisitor(&semanticsContext);
+
         auto argCount = argExprs.getCount();
         for(Index ii = 0; ii < argCount; ++ii )
         {
             auto argExpr = argExprs[ii];
             auto paramInfo = componentType->getSpecializationParam(ii);
 
-            // TODO: We should support non-type arguments here
-
-            auto argType = checkProperType(linkage, TypeExp(argExpr), sink);
-            if( !argType )
-            {
-                // If no witness was found, then we will be unable to satisfy
-                // the conformances required.
-                sink->diagnose(argExpr,
-                    Diagnostics::expectedAType,
-                    argExpr->type);
-                continue;
-            }
-
             SpecializationArg arg;
-            arg.val = argType;
+            arg.val = semanticsVisitor.ExtractGenericArgVal(argExpr);
             outArgs.add(arg);
         }
     }
@@ -1757,7 +1799,7 @@ static bool doesParameterMatch(
             RefPtr<Expr> argExpr;
             for (auto & s : scopesToTry)
             {
-                argExpr = linkage->parseTypeString(name, s);
+                argExpr = linkage->parseTermString(name, s);
                 argExpr = semantics.CheckTerm(argExpr);
                 if( argExpr )
                 {
@@ -1790,7 +1832,7 @@ static bool doesParameterMatch(
         SemanticsVisitor visitor(&sharedSemanticsContext);
 
         SpecializationParams specializationParams;
-        _collectExistentialSpecializationParamsRec(specializationParams, unspecializedType);
+        _collectExistentialSpecializationParamsRec(specializationParams, unspecializedType, SourceLoc());
 
         assert(specializationParams.getCount() == argCount);
 

source/slang/slang-compiler.cpp

@@ -2382,6 +2382,37 @@ SlangResult dissassembleDXILUsingDXC(
         BackEndCompileRequest* compileRequest,
         EndToEndCompileRequest* endToEndReq)
     {
+        // If we are about to generate output code, but we still
+        // have unspecialized generic/existential parameters,
+        // then there is a problem.
+        //
+        auto program = compileRequest->getProgram();
+        auto specializationParamCount = program->getSpecializationParamCount();
+        if( specializationParamCount != 0 )
+        {
+            auto sink = compileRequest->getSink();
+            
+            for( Index ii = 0; ii < specializationParamCount; ++ii )
+            {
+                auto specializationParam = program->getSpecializationParam(ii);
+                if( auto decl = as<Decl>(specializationParam.object) )
+                {
+                    sink->diagnose(specializationParam.loc, Diagnostics::specializationParameterOfNameNotSpecialized, decl);
+                }
+                else if( auto type = as<Type>(specializationParam.object) )
+                {
+                    sink->diagnose(specializationParam.loc, Diagnostics::specializationParameterOfNameNotSpecialized, type);
+                }
+                else
+                {
+                    sink->diagnose(specializationParam.loc, Diagnostics::specializationParameterNotSpecialized);
+                }
+            }
+
+            return;
+        }
+
+
         // Go through the code-generation targets that the user
         // has specified, and generate code for each of them.
         //

source/slang/slang-compiler.h

@@ -1206,7 +1206,7 @@ namespace Slang
         ///
         SlangResult loadFile(String const& path, PathInfo& outPathInfo, ISlangBlob** outBlob);
 
-        RefPtr<Expr> parseTypeString(String typeStr, RefPtr<Scope> scope);
+        RefPtr<Expr> parseTermString(String str, RefPtr<Scope> scope);
 
         Type* specializeType(
             Type*           unspecializedType,

source/slang/slang-decl-defs.h

@@ -180,6 +180,11 @@ END_SYNTAX_CLASS()
 SYNTAX_CLASS(GlobalGenericParamDecl, AggTypeDecl)
 END_SYNTAX_CLASS()
 
+// A `__generic_value_param` declaration, which defines an existential
+// value parameter (not a type parameter.
+SYNTAX_CLASS(GlobalGenericValueParamDecl, VarDeclBase)
+END_SYNTAX_CLASS()
+
 // A scope for local declarations (e.g., as part of a statement)
 SIMPLE_SYNTAX_CLASS(ScopeDecl, ContainerDecl)
 

source/slang/slang-diagnostic-defs.h

@@ -363,11 +363,16 @@ DIAGNOSTIC(38002, Note, entryPointCandidate, "see candidate declaration for entr
 DIAGNOSTIC(38003, Error, entryPointSymbolNotAFunction, "entry point '$0' must be declared as a function")
 
 DIAGNOSTIC(38004, Error, entryPointTypeParameterNotFound, "no type found matching entry-point type parameter name '$0'")
-DIAGNOSTIC(38005, Error, globalGenericArgumentNotAType, "argument for global generic parameter '$0' must be a type")
+DIAGNOSTIC(38005, Error, expectedTypeForSpecializationArg, "expected a type as argument for specialization parameter '$0'")
 
 DIAGNOSTIC(38006, Warning, specifiedStageDoesntMatchAttribute, "entry point '$0' being compiled for the '$1' stage has a '[shader(...)]' attribute that specifies the '$2' stage")
 DIAGNOSTIC(38007, Error, entryPointHasNoStage, "no stage specified for entry point '$0'; use either a '[shader(\"name\")]' function attribute or the '-stage <name>' command-line option to specify a stage")
 
+DIAGNOSTIC(38008, Error, specializationParameterOfNameNotSpecialized, "no specialization argument was provided for specialization parameter '$0'")
+DIAGNOSTIC(38008, Error, specializationParameterNotSpecialized, "no specialization argument was provided for specialization parameter")
+
+DIAGNOSTIC(38009, Error, expectedValueOfTypeForSpecializationArg, "expected a constant value of type '$0' as argument for specialization parameter '$1'")
+
 DIAGNOSTIC(38100, Error, typeDoesntImplementInterfaceRequirement, "type '$0' does not provide required interface member '$1'")
 DIAGNOSTIC(38101, Error, thisExpressionOutsideOfTypeDecl, "'this' expression can only be used in members of an aggregate type")
 DIAGNOSTIC(38102, Error, initializerNotInsideType, "an 'init' declaration is only allowed inside a type or 'extension' declaration")
@@ -388,7 +393,6 @@ DIAGNOSTIC(38025, Error, mismatchSpecializationArguments, "expected $0 specializ
 DIAGNOSTIC(38026, Error, globalTypeArgumentDoesNotConformToInterface, "type argument `$1` for global generic parameter `$0` does not conform to interface `$2`.")
 
 DIAGNOSTIC(38027, Error, mismatchExistentialSlotArgCount, "expected $0 existential slot arguments ($1 provided)")
-DIAGNOSTIC(38028, Error, existentialSlotArgNotAType, "existential slot argument $0 was not a type")
 DIAGNOSTIC(38029, Error,typeArgumentDoesNotConformToInterface, "type argument '$0' does not conform to the required interface '$1'")
 
 DIAGNOSTIC(38200, Error, recursiveModuleImport, "module `$0` recursively imports itself")

source/slang/slang-ir-insts.h

@@ -1926,7 +1926,18 @@ struct IRBuilder
         UInt            caseArgCount,
         IRInst* const* caseArgs);
 
-    IRGlobalGenericParam* emitGlobalGenericParam();
+    IRGlobalGenericParam* emitGlobalGenericParam(
+        IRType* type);
+
+    IRGlobalGenericParam* emitGlobalGenericTypeParam()
+    {
+        return emitGlobalGenericParam(getTypeKind());
+    }
+
+    IRGlobalGenericParam* emitGlobalGenericWitnessTableParam()
+    {
+        return emitGlobalGenericParam(getWitnessTableType());
+    }
 
     IRBindGlobalGenericParam* emitBindGlobalGenericParam(
         IRInst* param,

source/slang/slang-ir-link.cpp

@@ -552,7 +552,7 @@ IRGlobalGenericParam* cloneGlobalGenericParamImpl(
     IRGlobalGenericParam*           originalVal,
     IROriginalValuesForClone const& originalValues)
 {
-    auto clonedVal = builder->emitGlobalGenericParam();
+    auto clonedVal = builder->emitGlobalGenericParam(originalVal->getFullType());
     cloneSimpleGlobalValueImpl(context, originalVal, originalValues, clonedVal);
     return clonedVal;
 }

source/slang/slang-ir.cpp

@@ -3328,12 +3328,13 @@ namespace Slang
         return inst;
     }
 
-    IRGlobalGenericParam* IRBuilder::emitGlobalGenericParam()
+    IRGlobalGenericParam* IRBuilder::emitGlobalGenericParam(
+        IRType* type)
     {
         IRGlobalGenericParam* irGenericParam = createInst<IRGlobalGenericParam>(
             this,
             kIROp_GlobalGenericParam,
-            nullptr);
+            type);
         addGlobalValue(this, irGenericParam);
         return irGenericParam;
     }

source/slang/slang-lower-to-ir.cpp

@@ -4211,7 +4211,7 @@ struct DeclLoweringVisitor : DeclVisitor<DeclLoweringVisitor, LoweredValInfo>
             // This is a constraint on a global generic type parameters,
             // and so it should lower as a parameter of its own.
 
-            auto inst = getBuilder()->emitGlobalGenericParam();
+            auto inst = getBuilder()->emitGlobalGenericWitnessTableParam();
             addLinkageDecoration(context, inst, decl);
             return LoweredValInfo::simple(inst);
         }
@@ -4227,11 +4227,21 @@ struct DeclLoweringVisitor : DeclVisitor<DeclLoweringVisitor, LoweredValInfo>
 
     LoweredValInfo visitGlobalGenericParamDecl(GlobalGenericParamDecl* decl)
     {
-        auto inst = getBuilder()->emitGlobalGenericParam();
+        auto inst = getBuilder()->emitGlobalGenericTypeParam();
         addLinkageDecoration(context, inst, decl);
         return LoweredValInfo::simple(inst);
     }
 
+    LoweredValInfo visitGlobalGenericValueParamDecl(GlobalGenericValueParamDecl* decl)
+    {
+        auto builder = getBuilder();
+        auto type = lowerType(context, decl->type);
+        auto inst = builder->emitGlobalGenericParam(type);
+        addLinkageDecoration(context, inst, decl);
+        return LoweredValInfo::simple(inst);
+    }
+
+
     void lowerWitnessTable(
         IRGenContext*                               subContext,
         WitnessTable*                               astWitnessTable,