Skip to content

Commit 54d3c5f

Browse files
author
Tim Foley
authored
Remove legacy feature for merging global shader parameters (shader-slang#1139)
* Remove legacy feature for merging global shader parameters There is a fair amount of special-case code in the Slang compiler today to deal with the scenario where a programmer declares the "same" shader parameter across two different translation units: ```hlsl // a.hlsl Texture2D a; cbuffer C { float4 c; } ``` ```hlsl // b.hlsl cbuffer C { float4 c; } Texture2D b; ``` An important note here is that the declaration of `C` may be in a header file that both `a.hlsl` and `b.hlsl` `#include`, because from the standpoint of the parser and later stages of the compiler, there is no difference between `C` being in an included file vs. it being copy-pasted across both `a.hlsl` and `b.hlsl`. When a user invokes `slangc a.hlsl b.hlsl` (or the equivalent via the API), then they may decide that it is "obvious" that the shader parameter `C` is the "same" in both `a.hlsl` and `b.hlsl`. Knowing that the parameter is the "same" may lead them to make certain assumptions: * They may assume that generated code for entry points in `a.hlsl` and `b.hlsl` will both agree on the exact `register`/`binding` occupied by `C`. * They may assume that reflection information for their program will only reflect `C` once, and it will reflect it in a way that is applicable to entry points in both `a.hlsl` and `b.hlsl` * They may assume that the compiler can and should handle this use case even when `C` contains fields with `struct` types that are declared in both `a.hlsl` and `b.hlsl` that have the "same" definition. * They may assume that in cases where `C` is declared inconsistently between `a.hlsl` and `b.hlsl` the compiler can and will diagnose an error. Making these assumptions work in practice required a lot of special-case code: * When composing/linking programs was `ComponentType`s we had to include a special case `LegacyProgram` type that could provide these "do what I mean" semantics, since they are *not* what one would want in the general case for a `CompositeComponentType`. * During enumeration of global shader parameter in a `LegacyProgram`, we had to detect parameters from distinct modules (translation units) with the same name, and then enforce that they must have the "same" type (via an ad hoc recursive structural type match). No other semantic checking logic needs or uses that kind of structural check. * During parameter binding generation, we need to handle the case where a single global shader parameter might have multiple declarations, and make sure to collect explicit bindings from all of them (checking for inconsistency) and also to apply generated bindings to all of them. * The `mapVarToLayout` member in `StructTypeLayout` is a concession to the fact that we might have multiple `VarDecl`s for each field of the struct that represents the global scope, we might need to look up a field and its layout using any of those declarations (much of the need for this field had gone away now that IR passes are largely using IR-based layout). All of these different special cases added more complex code in many places in the compiler, all to support a scenario that isn't especially common. Most users won't be affected by the original issue, because they will do one of several things that rule it out: * Anybody using `slangc` like a stand-in for `fxc` or `dxc` and compiling one translation unit at a time will not suffer from any problems. If/when such users want consistent bindings across translation units, they already use either explicit binding or rely on consistent ordering and implicit binding. * Anybody who puts all the entry points that get combined into a pass/pipeline in a single file will not have problems. They will automatically get consistent bindings because of Slang's guarantees, and there can't be duplicated declarations when there is only one translation unit. * Anybody using `import` to factor out common declarations while compiling multiple translation units at once will not be affected. Parameters declared in an `import`ed module are the "same" in a much deeper way that it is trivial for Slang to support. Only users of the Falcor framework are likely to be affected by this, and they have two easy migration paths: either put related entry points into the same file, or factor common parameters into an `import`ed module. (It is also worth noting that for command-line `slangc`, it is possible to have a single module with multiple `.slang` files in it, which can all see global declarations like parameters across all the files. Anybody who buys into doing things the Slang Way should have no problem avoiding duplicated declarations) With the rationale out of the way, the actual change mostly just amounts to deleting lots of code that is no longer needed. An astute reviewer might notice several `assert`-fail conditions where complex Slang features were never actually made to work correctly with this legacy behavior. A small number of test cases broke with the code changes, but these were tests that specifically exercised the behavior being removed. In the case of the tests around binding/reflection generating, I rewrote the tests to use one of the idomatic workarounds (putting the shared parameters into an `import`ed module), but doing so required me to add support for `#include` when doing pass-through compilation with `fxc`. That logic added a bit more cruft than I had originally hoped to this commit, but having `#include` support when doing pass-through compilation is probably a net win. * fixup: 64-bit warning
1 parent 4e2cfc9 commit 54d3c5f

17 files changed

+489
-1226
lines changed

source/slang/slang-check-shader.cpp

+14-499
Large diffs are not rendered by default.

source/slang/slang-compiler.cpp

+104-2
Original file line numberDiff line numberDiff line change
@@ -939,6 +939,93 @@ namespace Slang
939939
return UnownedStringSlice();
940940
}
941941

942+
/// Read a file in the context of handling a preprocessor directive
943+
static SlangResult readFile(
944+
Linkage* linkage,
945+
String const& path,
946+
ISlangBlob** outBlob)
947+
{
948+
// The actual file loading will be handled by the file system
949+
// associated with the parent linkage.
950+
//
951+
auto fileSystemExt = linkage->getFileSystemExt();
952+
SLANG_RETURN_ON_FAIL(fileSystemExt->loadFile(path.getBuffer(), outBlob));
953+
954+
return SLANG_OK;
955+
}
956+
957+
struct FxcIncludeHandler : ID3DInclude
958+
{
959+
Linkage* linkage;
960+
DiagnosticSink* sink;
961+
IncludeHandler* includeHandler;
962+
PathInfo rootPathInfo;
963+
964+
STDMETHOD(Open)(D3D_INCLUDE_TYPE IncludeType, LPCSTR pFileName, LPCVOID pParentData, LPCVOID *ppData, UINT *pBytes) override
965+
{
966+
SLANG_UNUSED(IncludeType);
967+
SLANG_UNUSED(pParentData);
968+
969+
String path(pFileName);
970+
971+
SourceLoc loc;
972+
973+
PathInfo includedFromPathInfo = rootPathInfo;
974+
975+
if (!includeHandler)
976+
{
977+
return SLANG_E_NOT_IMPLEMENTED;
978+
}
979+
980+
// Find the path relative to the foundPath
981+
PathInfo filePathInfo;
982+
if (SLANG_FAILED(includeHandler->findFile(path, includedFromPathInfo.foundPath, filePathInfo)))
983+
{
984+
return SLANG_E_CANNOT_OPEN;
985+
}
986+
987+
// We must have a uniqueIdentity to be compare
988+
if (!filePathInfo.hasUniqueIdentity())
989+
{
990+
return SLANG_E_ABORT;
991+
}
992+
993+
// Simplify the path
994+
filePathInfo.foundPath = includeHandler->simplifyPath(filePathInfo.foundPath);
995+
996+
// See if this an already loaded source file
997+
auto sourceManager = linkage->getSourceManager();
998+
SourceFile* sourceFile = sourceManager->findSourceFileRecursively(filePathInfo.uniqueIdentity);
999+
1000+
// If not create a new one, and add to the list of known source files
1001+
if (!sourceFile)
1002+
{
1003+
ComPtr<ISlangBlob> foundSourceBlob;
1004+
if (SLANG_FAILED(readFile(linkage, filePathInfo.foundPath, foundSourceBlob.writeRef())))
1005+
{
1006+
return SLANG_E_CANNOT_OPEN;
1007+
}
1008+
1009+
sourceFile = sourceManager->createSourceFileWithBlob(filePathInfo, foundSourceBlob);
1010+
sourceManager->addSourceFile(filePathInfo.uniqueIdentity, sourceFile);
1011+
}
1012+
1013+
// This is a new parse (even if it's a pre-existing source file), so create a new SourceUnit
1014+
SourceView* sourceView = sourceManager->createSourceView(sourceFile, &filePathInfo);
1015+
1016+
*ppData = sourceView->getContent().begin();
1017+
*pBytes = (UINT) sourceView->getContentSize();
1018+
1019+
return S_OK;
1020+
}
1021+
1022+
STDMETHOD(Close)(LPCVOID pData) override
1023+
{
1024+
SLANG_UNUSED(pData);
1025+
return S_OK;
1026+
}
1027+
};
1028+
9421029
SlangResult emitDXBytecodeForEntryPoint(
9431030
BackEndCompileRequest* compileRequest,
9441031
EntryPoint* entryPoint,
@@ -963,6 +1050,8 @@ namespace Slang
9631050

9641051
auto profile = getEffectiveProfile(entryPoint, targetReq);
9651052

1053+
auto linkage = compileRequest->getLinkage();
1054+
9661055
// If we have been invoked in a pass-through mode, then we need to make sure
9671056
// that the downstream compiler sees whatever options were passed to Slang
9681057
// via the command line or API.
@@ -971,6 +1060,14 @@ namespace Slang
9711060
//
9721061
List<D3D_SHADER_MACRO> dxMacrosStorage;
9731062
D3D_SHADER_MACRO const* dxMacros = nullptr;
1063+
1064+
IncludeHandlerImpl includeHandler;
1065+
includeHandler.linkage = linkage;
1066+
includeHandler.searchDirectories = &linkage->searchDirectories;
1067+
1068+
FxcIncludeHandler fxcIncludeHandlerStorage;
1069+
FxcIncludeHandler* fxcIncludeHandler = nullptr;
1070+
9741071
if(auto translationUnit = findPassThroughTranslationUnit(endToEndReq, entryPointIndex))
9751072
{
9761073
for( auto& define : translationUnit->compileRequest->preprocessorDefinitions )
@@ -991,6 +1088,12 @@ namespace Slang
9911088
dxMacrosStorage.add(nullTerminator);
9921089

9931090
dxMacros = dxMacrosStorage.getBuffer();
1091+
1092+
fxcIncludeHandler = &fxcIncludeHandlerStorage;
1093+
fxcIncludeHandler->linkage = linkage;
1094+
fxcIncludeHandler->sink = compileRequest->getSink();
1095+
fxcIncludeHandler->includeHandler = &includeHandler;
1096+
fxcIncludeHandler->rootPathInfo = translationUnit->m_sourceFiles[0]->getPathInfo();
9941097
}
9951098

9961099
DWORD flags = 0;
@@ -1018,7 +1121,6 @@ namespace Slang
10181121
flags |= D3DCOMPILE_ENABLE_STRICTNESS;
10191122
flags |= D3DCOMPILE_ENABLE_UNBOUNDED_DESCRIPTOR_TABLES;
10201123

1021-
auto linkage = compileRequest->getLinkage();
10221124
switch( linkage->optimizationLevel )
10231125
{
10241126
default:
@@ -1049,7 +1151,7 @@ namespace Slang
10491151
hlslCode.getLength(),
10501152
sourcePath.getBuffer(),
10511153
dxMacros,
1052-
nullptr,
1154+
fxcIncludeHandler,
10531155
getText(entryPoint->getName()).begin(),
10541156
GetHLSLProfileName(profile).getBuffer(),
10551157
flags,

source/slang/slang-compiler.h

+26-90
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010

1111
#include "slang-diagnostics.h"
1212
#include "slang-name.h"
13+
#include "slang-preprocessor.h"
1314
#include "slang-profile.h"
1415
#include "slang-syntax.h"
1516

@@ -159,20 +160,6 @@ namespace Slang
159160
Int specializationParamCount = 0;
160161
};
161162

162-
/// Extended information specific to global shader parameters
163-
struct GlobalShaderParamInfo : ShaderParamInfo
164-
{
165-
// TODO: This type should be eliminated if/when we remove
166-
// support for compilation with multiple translation units
167-
// that all declare the "same" shader parameter (e.g., a
168-
// `cbuffer`) and expect those duplicate declarations
169-
// to get the same parameter binding/layout.
170-
171-
// Additional global-scope declarations that are conceptually
172-
// declaring the "same" parameter as the `paramDeclRef`.
173-
List<DeclRef<VarDeclBase>> additionalParamDeclRefs;
174-
};
175-
176163
/// A request for the front-end to find and validate an entry-point function
177164
struct FrontEndEntryPointRequest : RefObject
178165
{
@@ -321,7 +308,7 @@ namespace Slang
321308
virtual Index getShaderParamCount() = 0;
322309

323310
/// Get one of the global shader parametesr linked into this component type.
324-
virtual GlobalShaderParamInfo getShaderParam(Index index) = 0;
311+
virtual ShaderParamInfo getShaderParam(Index index) = 0;
325312

326313
/// Get the number of (unspecialized) specialization parameters for the component type.
327314
virtual Index getSpecializationParamCount() = 0;
@@ -351,8 +338,7 @@ namespace Slang
351338
/// to the provided `sink`.
352339
///
353340
/// TODO: This function shouldn't be on the base class, since
354-
/// it only really makes sense on `Module` and (as a compatibility
355-
/// feature) on `LegacyProgram`.
341+
/// it only really makes sense on `Module`.
356342
///
357343
Type* getTypeFromString(
358344
String const& typeStr,
@@ -508,7 +494,7 @@ namespace Slang
508494
String getEntryPointMangledName(Index index) SLANG_OVERRIDE;
509495

510496
Index getShaderParamCount() SLANG_OVERRIDE;
511-
GlobalShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE;
497+
ShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE;
512498

513499
Index getSpecializationParamCount() SLANG_OVERRIDE;
514500
SpecializationParam const& getSpecializationParam(Index index) SLANG_OVERRIDE;
@@ -555,7 +541,7 @@ namespace Slang
555541
//
556542
List<EntryPoint*> m_entryPoints;
557543
List<String> m_entryPointMangledNames;
558-
List<GlobalShaderParamInfo> m_shaderParams;
544+
List<ShaderParamInfo> m_shaderParams;
559545
List<SpecializationParam> m_specializationParams;
560546
List<ComponentType*> m_requirements;
561547

@@ -595,7 +581,7 @@ namespace Slang
595581
String getEntryPointMangledName(Index index) SLANG_OVERRIDE;
596582

597583
Index getShaderParamCount() SLANG_OVERRIDE { return m_base->getShaderParamCount(); }
598-
GlobalShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE { return m_base->getShaderParam(index); }
584+
ShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE { return m_base->getShaderParam(index); }
599585

600586
Index getSpecializationParamCount() SLANG_OVERRIDE { return 0; }
601587
SpecializationParam const& getSpecializationParam(Index index) SLANG_OVERRIDE { SLANG_UNUSED(index); static SpecializationParam dummy; return dummy; }
@@ -726,7 +712,7 @@ namespace Slang
726712
String getEntryPointMangledName(Index index) SLANG_OVERRIDE;
727713

728714
Index getShaderParamCount() SLANG_OVERRIDE { return 0; }
729-
GlobalShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE { SLANG_UNUSED(index); return GlobalShaderParamInfo(); }
715+
ShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE { SLANG_UNUSED(index); return ShaderParamInfo(); }
730716

731717
class EntryPointSpecializationInfo : public SpecializationInfo
732718
{
@@ -890,7 +876,7 @@ namespace Slang
890876
String getEntryPointMangledName(Index index) SLANG_OVERRIDE { SLANG_UNUSED(index); return String(); }
891877

892878
Index getShaderParamCount() SLANG_OVERRIDE { return m_shaderParams.getCount(); }
893-
GlobalShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE { return m_shaderParams[index]; }
879+
ShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE { return m_shaderParams[index]; }
894880

895881
Index getSpecializationParamCount() SLANG_OVERRIDE { return m_specializationParams.getCount(); }
896882
SpecializationParam const& getSpecializationParam(Index index) SLANG_OVERRIDE { return m_specializationParams[index]; }
@@ -940,7 +926,7 @@ namespace Slang
940926
// The IR for the module
941927
RefPtr<IRModule> m_irModule = nullptr;
942928

943-
List<GlobalShaderParamInfo> m_shaderParams;
929+
List<ShaderParamInfo> m_shaderParams;
944930
SpecializationParams m_specializationParams;
945931

946932
List<Module*> m_requirements;
@@ -1479,64 +1465,6 @@ namespace Slang
14791465
List<RefPtr<ComponentType>> m_unspecializedEntryPoints;
14801466
};
14811467

1482-
/// A "legacy" program composes multiple translation units from a single compile request,
1483-
/// and takes care to treat global declarations of the same name from different translation
1484-
/// units as representing the "same" parameter.
1485-
///
1486-
/// TODO: This type only exists to support a single requirement: that multiple translation
1487-
/// units can be compiled in one pass and be guaranteed that the "same" parameter declared
1488-
/// in different translation units (hence different modules) will get the same layout.
1489-
/// This feature should be deprecated and removed as soon as possible, since the complexity
1490-
/// it creates in the codebase is not justified by its limited utility.
1491-
///
1492-
class LegacyProgram : public ComponentType
1493-
{
1494-
public:
1495-
LegacyProgram(
1496-
Linkage* linkage,
1497-
List<RefPtr<TranslationUnitRequest>> const& translationUnits,
1498-
DiagnosticSink* sink);
1499-
1500-
Index getTranslationUnitCount() { return m_translationUnits.getCount(); }
1501-
RefPtr<TranslationUnitRequest> getTranslationUnit(Index index) { return m_translationUnits[index]; }
1502-
1503-
Index getEntryPointCount() SLANG_OVERRIDE { return 0; }
1504-
RefPtr<EntryPoint> getEntryPoint(Index index) SLANG_OVERRIDE { SLANG_UNUSED(index); return nullptr; }
1505-
String getEntryPointMangledName(Index index) SLANG_OVERRIDE { SLANG_UNUSED(index); return String(); }
1506-
1507-
Index getShaderParamCount() SLANG_OVERRIDE { return m_shaderParams.getCount(); }
1508-
GlobalShaderParamInfo getShaderParam(Index index) SLANG_OVERRIDE { return m_shaderParams[index]; }
1509-
1510-
Index getSpecializationParamCount() SLANG_OVERRIDE { return m_specializationParams.getCount(); }
1511-
SpecializationParam const& getSpecializationParam(Index index) SLANG_OVERRIDE { return m_specializationParams[index]; }
1512-
1513-
Index getRequirementCount() SLANG_OVERRIDE;
1514-
RefPtr<ComponentType> getRequirement(Index index) SLANG_OVERRIDE;
1515-
1516-
List<Module*> const& getModuleDependencies() SLANG_OVERRIDE { return m_moduleDependencies.getModuleList(); }
1517-
List<String> const& getFilePathDependencies() SLANG_OVERRIDE { return m_fileDependencies.getFilePathList(); }
1518-
1519-
protected:
1520-
void acceptVisitor(ComponentTypeVisitor* visitor, SpecializationInfo* specializationInfo) SLANG_OVERRIDE;
1521-
1522-
RefPtr<SpecializationInfo> _validateSpecializationArgsImpl(
1523-
SpecializationArg const* args,
1524-
Index argCount,
1525-
DiagnosticSink* sink) SLANG_OVERRIDE;
1526-
1527-
private:
1528-
void _collectShaderParams(DiagnosticSink* sink);
1529-
1530-
List<RefPtr<TranslationUnitRequest>> m_translationUnits;
1531-
1532-
List<EntryPoint*> m_entryPoints;
1533-
List<GlobalShaderParamInfo> m_shaderParams;
1534-
List<ComponentType*> m_requirements;
1535-
SpecializationParams m_specializationParams;
1536-
ModuleDependencyList m_moduleDependencies;
1537-
FilePathDependencyList m_fileDependencies;
1538-
};
1539-
15401468
/// A visitor for use with `ComponentType`s, allowing dispatch over the concrete subclasses.
15411469
class ComponentTypeVisitor
15421470
{
@@ -1552,23 +1480,14 @@ namespace Slang
15521480
virtual void visitModule(Module* module, Module::ModuleSpecializationInfo* specializationInfo) = 0;
15531481
virtual void visitComposite(CompositeComponentType* composite, CompositeComponentType::CompositeSpecializationInfo* specializationInfo) = 0;
15541482
virtual void visitSpecialized(SpecializedComponentType* specialized) = 0;
1555-
virtual void visitLegacy(LegacyProgram* legacy, CompositeComponentType::CompositeSpecializationInfo* specializationInfo) = 0;
15561483

15571484
protected:
15581485
// These helpers can be used to recurse into the logical children of a
15591486
// component type, and are useful for the common case where a visitor
15601487
// only cares about a few leaf cases.
15611488
//
1562-
// Note that for a `LegacyProgram` the "children" in this case are the
1563-
// `Module`s of the translation units that make up the legacy program.
1564-
// In some cases this is what is desired, but in others it is incorrect
1565-
// to treat a legacy program as a composition of modules, and instead
1566-
// it should be treated directly as a leaf case. Clients should make
1567-
// an informed decision based on an understanding of what `LegacyProgram` is used for.
1568-
//
15691489
void visitChildren(CompositeComponentType* composite, CompositeComponentType::CompositeSpecializationInfo* specializationInfo);
15701490
void visitChildren(SpecializedComponentType* specialized);
1571-
void visitChildren(LegacyProgram* legacy, CompositeComponentType::CompositeSpecializationInfo* specializationInfo);
15721491
};
15731492

15741493
/// A `TargetProgram` represents a `ComponentType` specialized for a particular `TargetRequest`
@@ -2079,6 +1998,23 @@ namespace Slang
20791998
PassThroughMode m_defaultDownstreamCompilers[int(SourceLanguage::CountOf)];
20801999
};
20812000

2001+
struct IncludeHandlerImpl : IncludeHandler
2002+
{
2003+
Linkage* linkage;
2004+
SearchDirectoryList* searchDirectories;
2005+
2006+
ISlangFileSystemExt* _getFileSystemExt();
2007+
2008+
SlangResult _findFile(SlangPathType fromPathType, const String& fromPath, const String& path, PathInfo& pathInfoOut);
2009+
2010+
virtual SlangResult findFile(
2011+
String const& pathToInclude,
2012+
String const& pathIncludedFrom,
2013+
PathInfo& pathInfoOut) override;
2014+
2015+
virtual String simplifyPath(const String& path) override;
2016+
};
2017+
20822018

20832019
//
20842020
// The following functions are utilties to convert between

source/slang/slang-lower-to-ir.cpp

-11
Original file line numberDiff line numberDiff line change
@@ -6941,17 +6941,6 @@ struct SpecializedComponentTypeIRGenContext : ComponentTypeVisitor
69416941
{
69426942
visitChildren(specialized);
69436943
}
6944-
6945-
void visitLegacy(LegacyProgram* legacy, CompositeComponentType::CompositeSpecializationInfo* specializationInfo) SLANG_OVERRIDE
6946-
{
6947-
// TODO: This case should be akin to the `Module` case,
6948-
// and deal with global-scope specialization parameters
6949-
// directly.
6950-
//
6951-
SLANG_UNUSED(legacy);
6952-
SLANG_UNUSED(specializationInfo);
6953-
SLANG_UNIMPLEMENTED_X("legacy program case");
6954-
}
69556944
};
69566945

69576946
RefPtr<IRModule> generateIRForSpecializedComponentType(

0 commit comments

Comments
 (0)