Skip to main content

Browse the archive

https://github.com/shader-slang/slang
06 April 2024, 09:30:39 UTC
sort by:
Revision Author Date Message Commit Date
21a14cb Tim Foley 06 July 2017, 16:52:53 UTC Merge pull request #53 from tfoleyNV/cross-compilation Initial work on cross-compilation 06 July 2017, 16:52:53 UTC
f145e09 Tim Foley 28 June 2017, 20:34:38 UTC Start to support cross-compilation via "lowering" pass - The big change here is the introduction of a "lowering" pass that takes an input AST from the semantic checker, and produces an output AST suitable for emitting. The intention is that he lowering pass is responsible for: - Stripping out unused code (when we have enough information to do so), by only outputting declarations that are transitively references from an entry point - When cross-compiling to GLSL, generating a suitable `void main()` entry point to wrap the user-written entry-point function - (Eventually) legalizing types in the program, by scalarizing aggregate types that mix uniform and resource types - (Eventually) instantiating generic declarations so that the resulting code only deals with fully specialized declarations - (Eventually) de-sugaring OOP constructs into basic "structs and functions" form - (Eventually) instantiating code that depends on interface types at the concrete types chosen - It is clear that there is still a lot of work to be done there, to this change is really about getting infrastructure in place without breaking the existing test cases. - One cleanup here is that we get rid of the idea of whole-translation-unit output, since that was specific to HLSL output, and there is really no strong reason for keeping it. Users should now just ask for the output for each entry point that they wanted to generate. - The biggest source of complexity for the lowering process is that it needs to produce the same AST structure as the input, to deal with the complexity of the rewriter case. That is, we need the output to be able to reproduce the input exactly in the case where we are rewriting and nothing needs to change, so the output format needs at least the degrees of freedom of the input. - As a result, we end up having to distinguish "rewriter" and "full" modes in both lowering and code-emit steps, so that we can react appropriately. - Generating a GLSL `main()` also adds a lot of complexity. Right now I'm using the simplest approach, where we always output the Slang/HLSL entry point as an ordinary function (as written) and then emit a simple GLSL `main()` to call it. I generate globals for all the shader inputs/outputs (these need to be scalarized and have explicit `location`s attached), and then collect these into the `struct` types of the original parameters as needed. - This approach will start to have some major down-sides once we have to deal with "arrayed" input/output - A long-term question here is how to replace entry-point parameter types with scalarized and/or "transposed" versions, while still letting the original code work as written (including copying those inputs to temporary arrays) - Split `BlockStatementSyntaxNode` into: - `BlockStmt` which just provides a scope around a `body` statement - `SeqStmt` which just allows multiple statements to be treated as one - Change how we emit `for` loops, to deal with the case where the initialization part might expand into multiple statements - Basically `for(A;B;C) {D}` becomes `{A; for(;B;C) {D}}`, so we can handle arbitrary statements for `A` - As an additional wrinkle, when we are rewriting HLSL, we just generate `A; for(;B;C) {D}` to deal with the broken scoping there - This change is needed because the lowering pass was sometimes expanding the original initialization statement `A` into a block `{A}`. Certainly if it declared multiple variables we'd need to handle it, and this seemed the easiest way - A more significant challenge for lowering would come if/when we ever wanted to support true short-circuiting behavior for `&&` and `||` - For right now I'm not changing the behavior of the "rewriter" mode, so we still have `UnparsedStmt` instances being generated, but it is clear that eventually we need to parse *all* input, even if we can't type-check 100% of it. This is required so that we can rewrite user code that might refer to a shader input with interface type. 06 July 2017, 16:17:04 UTC
c0d2c17 Tim Foley 28 June 2017, 20:34:21 UTC Add initial cross-compilation test case. 30 June 2017, 20:39:30 UTC
f313df3 Tim Foley 30 June 2017, 20:12:50 UTC Merge pull request #52 from tfoleyNV/syntax-meta Add meta-definitions for AST types 30 June 2017, 20:12:50 UTC
cab694d Tim Foley 29 June 2017, 15:55:09 UTC Add meta-definitions for AST types - The big change here is that all the definitions for syntax-node classes have been macro-ized, to that we can do light metaprogramming over them - The use of macros for this has big down-sides, but I'm not quite ready to do anything more heavy-weight right now - The macro-ized definitions can be included multiple times, to generate different declarations/code as needed - The first example of using this meta-programming facility is a new visitor system - The actual visitor base classes and the dispatch logic are all generated from the meta-files - There was only one visitor left in the code: the semantics checker, so that was ported to the new system. - All current test cases pass, so *of course* that means all is well. 30 June 2017, 19:22:33 UTC
b2b0867 Tim Foley 29 June 2017, 21:16:38 UTC Merge pull request #51 from tfoleyNV/refptr-string-cleanup Overhaul `RefPtr` and `String` 29 June 2017, 21:16:38 UTC
f4d900d Tim Foley 29 June 2017, 18:50:55 UTC Overhaul `RefPtr` and `String` - `RefPtr` no longer tries to have distinct cases for interal-vs-external reference counts. Instead we always require an internal reference count. - Types the used `RefPtr` but weren't `RefObject` were made to inherit `RefObject` - The `ReferenceCounted` base class was removed, so that only `RefObject` remains - Implicit conversion from `RefPtr<T>` to `T*` added - This created some complicates for other types that relied on implicit conversions, so this isn't a net cleanup right now - The main type that got messed up by the above was `String`, which previously held a `RefPtr<char, ...>`. This change thus *also* includes a major overhaul of `String`: - `String` now holds all its data via indirection, using a `StringRepresentation` that is a `RefObject`. This object holds a length, capacity, and directly stores the character data in its allocation. This means that `sizeof(String)==sizeof(void*)` - It is now possible to directly mutate a `String` by appending to its representation (we just need to ensure it has a reference count of `1`, possibly by cloning it). This means that `StringBuilder` is now basically just an idomatic use of `String` - A couple operations that just return sub-ranges of a `String` now return `StringSlice` to avoid allocation when it isn't needed. This required more work. - Indices into strings changed from `int` to `UInt` (which is pointer-sized). This had a bunch of follow-on changes because the value `-1` sometimes needs to be special-cased in code that uses indices. Further cleanups are probably needed here. 29 June 2017, 20:18:32 UTC
16613ed Tim Foley 28 June 2017, 19:17:47 UTC Merge pull request #50 from tfoleyNV/literal-fix-fix Store integer literals at high precision in AST 28 June 2017, 19:17:47 UTC
ff53669 Tim Foley 28 June 2017, 18:39:40 UTC Store integer literals at high precision in AST The lexer was creating an `unsigned long long` value, and then the AST was storing it in an `int`. This change makes both use a `long long`. This is obviously still a stopgap until I can get arbitrary precisions in here. 28 June 2017, 18:51:23 UTC
b8e3168 Tim Foley 28 June 2017, 18:26:02 UTC Merge pull request #48 from tfoleyNV/literal-suffix-fix Fix handling of literal suffixes 28 June 2017, 18:26:02 UTC
d601921 Tim Foley 28 June 2017, 17:20:16 UTC Actually respect suffixes on numeric literals. - Add logic to extract the value and suffix from a numeric literal - This duplicates some of the lexing logic, but this is hard to avoid without redundant runtime work - Note that I'm not using and stdlib string-to-number code. This should be more robust once it is working, but it is obviously error prone in the near term. The main up-sides to this are: - We can handle binary integer literals - We can handle hexadecimal floating-point literals without stdlib support - We can hypothetically support digit separators, if we ever wanted - The parser looks at the suffix characters sliced off by the lexer, and tries to pick a type to use for a literal - It uses `NULL` if there is no suffix, to avoid some nasty order dependencies where the stdlib might need to parse a number before it has seen the definition of `int` - Right now I only handle a few cases, so there may be bugs lurking here - The emit logic needs to handle the fact that a literal node in the AST might have a non-default type attached. - Right now I just quickly check for the most likely types, and emit the literal with a matching suffix. This doesn't seem robust if any source language supports a suffix for a type where a target has no corresponding suffix. In the long term some amount of casting is probably required. 28 June 2017, 18:08:03 UTC
4b3936e Tim Foley 28 June 2017, 15:49:04 UTC Rename literal tokens. These had a typo (`Literial`), so they needed a fix eventually. I also went ahead and made things a bit more verbose (`IntegerLiteral`, `FloatingPointLiteral`) because these names don't get used often enough for the brevity to pay off. 28 June 2017, 15:49:04 UTC
afe41a6 Tim Foley 27 June 2017, 21:43:53 UTC Merge pull request #47 from tfoleyNV/import-subdir Allow "dotted" import paths 27 June 2017, 21:43:53 UTC
9eee117 Tim Foley 27 June 2017, 21:22:01 UTC Allow "dotted" import paths The code: __import foo.bar; will try to import from a file matching "foo/bar.slang". I also went ahead and allowed a raw string literal in and import: __import "foo/bar"; (In the latter case, an explicit `/` must be used instead of `.`) 27 June 2017, 21:22:01 UTC
97f2531 Tim Foley 27 June 2017, 18:13:21 UTC Merge pull request #46 from tfoleyNV/import-exported Allow for re-export of an `import` declaration 27 June 2017, 18:13:21 UTC
31293d6 Tim Foley 27 June 2017, 17:24:25 UTC Allow for re-export of an `import` declaration If module `A.slang` contains `__exported __import B;` then any declarations from `B.slang` will be visible to any client code that does `__import A;`. This allows a user to make a single "umbrella" file that encompases a bunch of code files. Note that this really only affects scoping during Slang compilation/checking; at code generation time everything always gets emitted as raw HLSL/GLSL so that names will be visible whether we want them to be or not. 27 June 2017, 17:24:25 UTC
1716208 Tim Foley 27 June 2017, 16:36:19 UTC Merge pull request #45 from tfoleyNV/emit-imported-parameters Emit global-scope parameters from imported files. 27 June 2017, 16:36:19 UTC
996d4dc Tim Foley 27 June 2017, 16:13:19 UTC Emit global-scope parameters from imported files. This fixes up a bug in the earlier change that provided reflection for imported parameters; I'd failed to confirm that the code generation logic can handle imported parameters correctly. The main fix was to have an `import` declaration automatically use the global-scope layout already determined, sine imported parameters will in general appear there. 27 June 2017, 16:13:19 UTC
8824b10 Tim Foley 26 June 2017, 21:31:31 UTC Merge pull request #44 from tfoleyNV/string-literal-parsing-fix Fix parsing of string literals. 26 June 2017, 21:31:31 UTC
7d97d42 Tim Foley 26 June 2017, 21:09:53 UTC Fix parsing of string literals. String literals can be used as part of attributes, but we lacked an actual AST representation for them. This change adds basic parsing for string literals, as well as emit logic for them. I also included a fix for parsing of chained right-associative operators. To test these fixes, I've re-enabled one of the HLSL tests I disabled a while back. It would be good to go through and see how many of those we can re-enable now. 26 June 2017, 21:14:06 UTC
0259ddb Tim Foley 26 June 2017, 20:47:08 UTC Merge pull request #43 from tfoleyNV/import-macros Add support for `#import` 26 June 2017, 20:47:08 UTC
f6cb66f Tim Foley 26 June 2017, 17:52:31 UTC Check for re-import at translation-unit level Previously the code checked for a duplicate `#import` using a data structure attached to the compile request, but this would fail for nested imports. It also wouldn't work for a combination of `#import` and `__import`. This change makes it so that we instead track a set of already-imported modules in the semantic checking visitor, which is instantiated once per translation unit. We also key this set on the actual module (AST) imported, rather than on path/name/whatever, so hopefully it will be robust to the same thing getting imported multiple ways. 26 June 2017, 19:17:49 UTC
6e99b81 Tim Foley 26 June 2017, 17:11:00 UTC Make `#import` work with preprocessor macros With this change, there is now a meaningful semantic difference between `__import` and `#import`. An `__import` compiles the target file in a fresh environment, only providing it any macro definitions passed via command line or API. Any macros defined in the imported file are not made visible at the import site. One can think of an `__import` as a bit like `using namespace` in C++. A `#import` will tokenize the input in the same preprocessor environment as the importing file, and any macros defined along the way will be visible in the parent file. It is a *bit* like a `#include` with two big differences: - The imported code is always parsed as Slang, and as its own module with default flags, etc. (so semantic checks are on even if we are in "rewriter" mode). It is pulled into the outer namespace just as for `__import`. - A given file will only get `#import`ed once for a translation unit, so it behaves a bit like there is an implicit `#pragma once` in the target file 26 June 2017, 19:06:54 UTC
7d3bfe4 Tim Foley 26 June 2017, 16:32:40 UTC Replace "auto-import" with `#import` Right now `#import` only differs from `#include` in that it takes a string literal for a file name instead of a raw identifier (to which `.slang` gets appended). The next step is to make `#import` respect preprocessor state, while `__import` doesn't. 26 June 2017, 19:06:54 UTC
3f316dc Tim Foley 26 June 2017, 19:04:54 UTC Merge pull request #42 from tfoleyNV/reflect-imported Reflect imported code 26 June 2017, 19:04:54 UTC
07f9b9e Tim Foley 26 June 2017, 18:37:09 UTC Add expected output file for test. This is the cause of the test failures on AppVeyor. 26 June 2017, 18:37:09 UTC
0df1cca Tim Foley 26 June 2017, 18:13:27 UTC typo 26 June 2017, 18:13:27 UTC
3609c40 Tim Foley 26 June 2017, 17:55:19 UTC Turn on some debug output for AppVeyor test failure. 26 June 2017, 17:55:19 UTC
c16fc84 Tim Foley 26 June 2017, 16:07:07 UTC Include imported code when generating reflection data - The basic idea is simple: be sure to enumerate code in `__import`ed modules when generating reflection info - Note that we don't currently allow an entry point to appear in an imported module, so we only consider globlal-scope parameters - Although there isn't currently a real implementation of namespacing, I went ahead and ensured that parameters in imported modules are treated as distinct from parameters in the user's code, even if they have the same name. 26 June 2017, 17:31:43 UTC
d506737 Tim Foley 22 June 2017, 22:32:42 UTC Merge pull request #39 from tfoleyNV/vector-constructors Standard library additions + supporting features 22 June 2017, 22:32:42 UTC
1eb5a1b Tim Foley 21 June 2017, 18:34:56 UTC Revamp definitions of texture `Load` and `GetDimensions` These are annoyingly subtle. 21 June 2017, 18:59:52 UTC
c92d06c Tim Foley 21 June 2017, 18:23:04 UTC Bug fix: correct attribute on `operator~` The operator was being declared as `IntrinsicOp::Not` when it should be `IntrinsicOp::BitNot` 21 June 2017, 18:59:52 UTC
bdfe06d Tim Foley 21 June 2017, 18:18:33 UTC Emit: Add support for `while` and `do {} while` statements These were being passed over by the emit logic because I didn't have tests that used them. 21 June 2017, 18:59:52 UTC
7f7864e Tim Foley 21 June 2017, 17:43:25 UTC Support texture `Gather*()` operations The catch with these operations is that they return a vector based on the scalar of the element type of the texture. That is, given `Texture2D<float> t` the operation `t.GatherRed(...)` should return a `float4`. The ideal way to solve this would use associated types, but we aren't there yet, so I am using extension declarations. An extension can "capture" the identity of the element type, like so: __generic<T, let N : int> __extension Texture2D<vector<T,N> > { ... } That extension will match `Texture2D<float3>` and correctly capture `T == float`, so that we can use it in other operations. Getting this working required a bunch of changes: - Actually emit the relevant extension declarations in the stdlib - Fix the parser to be able to parse `Texture2D<vector<T,N> >` (that is, a nested generic app). - I actually went ahead and significantly overhauled the expression parser while I was there, because I just couldn't deal with the existing code any longer. - Added support for general-case lookup to look through `__extension` declarations. I had logic in place to special-case this for looking up "constructors" but hadn't done anything for general member lookup yet. - This required some annoying holes to be punched through the layers, because lookup might need to invoke semantic analysis to ensure that an extension has been checked. - There is some first-pass code trying to support looking up a `typedef` nested inside the `vector` type. This is a nice idea in principle, but the problem is that the `Texture2D<T>` definition would be looking up `T.Element` and not `float4.Element`, and that means we'd need machinery for doing lookup *through* interface conformances for a type parameter like `T` The big gotcha here is that none of this logic applies to `Texture2D<float>` (the original case I mentioned) because I am matching vector types and not scalars. Matching scalars *should* be as easy as: __generic<T : __BuitlinScalarType> __extension Texture2D<T> { ... } But I'd need to confirm that interface constraints like that actually work, or else that extension would *also* apply to `Texture2D<float4>` and break everything. 21 June 2017, 18:59:51 UTC
5148c59 Tim Foley 20 June 2017, 21:38:50 UTC HLSL/Slang standard library additions - Vector constructors that take two vectors that add up to the target size (`float4(float2, float2)`) - I now realize I implemented the general case here, but there really is only the one case... - Geometry shader output stream types now have `Append()` and `RestartStrip()` methods 20 June 2017, 21:38:50 UTC
5170a56 Tim Foley 20 June 2017, 21:17:15 UTC Merge pull request #37 from tfoleyNV/falcor-work Falcor work 20 June 2017, 21:17:15 UTC
d852ad9 Tim Foley 20 June 2017, 20:57:46 UTC Only emit each `import`ed module once. If the user imports a module along more than one path, we need to make sure we don't emit the code twice. I handle this by keeping a set of already-emitted modules. Down the line, a more robust code generation strategy for non-"rewriter" use cases would be handling this at the per-declaration level, and this logic wouldn't really be needed. 20 June 2017, 20:57:46 UTC
4313db8 Tim Foley 20 June 2017, 20:57:20 UTC Add a useful source location to `typedef` declarations This was just missing logic in the parser. 20 June 2017, 20:57:20 UTC
3d29c90 Tim Foley 20 June 2017, 19:52:12 UTC Check `import` decls before all others. This helps ensure that we pull things into scope at the right time. 20 June 2017, 19:52:12 UTC
338232b Tim Foley 20 June 2017, 19:14:31 UTC Merge pull request #36 from tfoleyNV/entry-point-indexing Overhaul handling of entry points and translation units. 20 June 2017, 19:14:31 UTC
327f2b7 Tim Foley 20 June 2017, 15:11:27 UTC Overhaul handling of entry points and translation units. The main user-visible change here is that instead of `spAddTranslationUnitEntryPoint` we have `spAddEntryPoint`, to reflect that the list of entry points is "global" to a compile request. As a result, `spGetEntryPointSource` now only needs the entry point index, and not the translation unit index. There are a bunch more behind-the-scenes changes, though, reflecting a streamlining of the concepts related to compilation into a smaller number of classes. Now there is: - `Session` (unchanged) to manage the lifetimes of shared stuff like the stdlib - `CompileRequest` (merges in `CompileOptions`) to handle all the lifetime related to a single invocation of the compiler - `TranslationUnitRequest` (merges `TranslationUnitOptions`, `CompileUnit`) to represent a single translation unit ("module") that the user is trying to compile. This is a single file for HLSL/GLSL, but can be multiple files for Slang. - `EntryPointRequest` (merges `EntryPointOption` and a bit of `EntryPointResult`) to track a single entry point that the user is asking to compile (that entry point always comes from a single translation unit) A lot of functions used to take some combination of these and end up with really long signatures. I've given most of the objects "parent" pointers so that they can get back to all the context they need, so most functions don't need as many parameters. It may eventually be important to tease these apart again, in particular: - The code-generation side of things (the `*Result` types) might need to be pulled out in case we want to codegen multiple times from the same AST - Similarly, the layout stuff may also need to be pulled out, in case we want to lay things out multiple times with different rules. 20 June 2017, 18:54:35 UTC
40617db Tim Foley 20 June 2017, 16:54:38 UTC Merge pull request #35 from tfoleyNV/input-patch-type-fix Fix types for `InputPatch` and `OutputPatch` 20 June 2017, 16:54:38 UTC
c163c33 Tim Foley 20 June 2017, 16:24:31 UTC Fix types for `InputPatch` and `OutputPatch` Fixes #34. I'd declared these as if they were `InputPatch<T>`, but they are really `InputPatch<T,N>`. This change fixes the declarations, and makes these types no longer inherit from the contrived `BuiltinGenericType`. Instead they are more-or-less ordinary `DeclRefType`s using the same approach that `MatrixExpressionType` uses. 20 June 2017, 16:35:51 UTC
5b85b51 Tim Foley 19 June 2017, 21:42:18 UTC Merge pull request #32 from tfoleyNV/line-directives Line directives 19 June 2017, 21:42:18 UTC
110b0b3 Tim Foley 19 June 2017, 21:26:49 UTC Merge pull request #33 from tfoleyNV/preprocessor-bug-fixes Fixes for preprocessor conditionals that use macros 19 June 2017, 21:26:49 UTC
01cdb43 Tim Foley 19 June 2017, 21:26:41 UTC Merge pull request #31 from tfoleyNV/escaped-newlines Bug fix for newline escaping. 19 June 2017, 21:26:41 UTC
19906b6 Tim Foley 19 June 2017, 21:21:14 UTC Fixes for preprocessor conditionals that use macros The basic underlying problem here is that the preprocessor tries to linger at the end of an input stream until it is sure it is time to advance. An input stream can include raw input files, or the expansion of a macro or macro argument. This was originally done to deal with not getting good end-of-line tokens when in directives (that issue has been fixed), but it is now a legacy issue that should probably be removed (but I am wary of making such a sweeping change). The problem that arises is that some code depends on what the actual input stream is (e.g., when turning conditionals on/off), and so we need to be careful. The bugs that this change affects arise when a `#if` or `#elseif` conditional expression *ends* with a macro expansion: #define FOO 2 #if 2 == FOO ... #endif When we try to start the preprocessor conditional block the "active" stream is still the expansion of `FOO`, when we needed it to be the input file. We fix this for now by snapshotting the input stream at the start of the directive, but a better long term fix would be to fix up this weird end-of-input behavior. 19 June 2017, 21:25:17 UTC
e65d2ad Tim Foley 19 June 2017, 21:13:55 UTC Bug fix for newline escaping. The previous changes had left out logic for "scrubbing" a token value that includes an escaped newline, because I expected it would only occur within whitespace. Unfortunately, some user code looked like this: ``` a + b ``` That is, there was a token at the very start of the line, after the escaped newline. As a result, after consuming the leading whitespace (which didn't end up consuming the escaped newline - but we could consider making it do so in future), the lexer started to lex a token that *starts* with an escaped newline, but turns out to be an identifer (which gets an invalid name). This change adds some ad-hoc code to "scrub" the value of *every* token, which wasteful but at least solves the problem. 19 June 2017, 21:13:55 UTC
947cb6a Tim Foley 19 June 2017, 21:01:34 UTC Emit `#line` directives more aggressively We now output a line directive for (nearly) every declaration, statement, and modifier, so that hopefully there will be fewer cases where a downstream error doesn't point to the correct line. This exposes a lot of issues where we can/should clean up the simplicity of the code we emit (e.g., not output redundant parens; tracking source locations for types better). These kinds of issues will need to be addressed in follow-on changes. A few big ones: - Because GLSL doesn't allow for file names in `#line` directives, we really need to expose some data that can clean up error messages (or can be used by an application to do the same) so that they know which file is which. - We really need a command line option (and an equivalent API flag) to turn off emission of `#line` directies, so that the user can get moderately clean code as output. 19 June 2017, 21:10:41 UTC
411b243 Tim Foley 19 June 2017, 20:20:53 UTC Don't emit redundant `#line` directives If the line number for the next token is within a small range, then go ahead and output newlines to get caught up, rather than emit a `#line` directive. This saves a small amount of clutter, and in the particular case where the number of lines is 1, it stops our current behavior of putting a directive on each line. 19 June 2017, 20:20:53 UTC
838e833 Tim Foley 19 June 2017, 18:09:23 UTC Merge pull request #30 from tfoleyNV/slang-checking-fix Make sure that semantic checks always apply to Slang files 19 June 2017, 18:09:23 UTC
91d796d Tim Foley 19 June 2017, 17:37:09 UTC Make sure that semantic checks always apply to Slang files That is, even if hte user specified the `-no-checking` option (or the equivalent via API), we still want/need to apply full semantic checks to Slang code, so that cross-compilation will be possible. 19 June 2017, 17:37:09 UTC
bb9ca29 Tim Foley 19 June 2017, 17:23:16 UTC Merge pull request #29 from tfoleyNV/auto-import Allow for automatic importing of Slang code 19 June 2017, 17:23:16 UTC
634522d Tim Foley 19 June 2017, 15:47:23 UTC Allow for automatic importing of Slang code The basic idea of this change is that user code can just write: #include "foo.h" and then if `foo.h` gets found in a list of registered directories for "auto-import," then it actually gets interpreted as if the user had writte, more or less: __import foo; That is, the code in `foo.h` will be treated as Slang, and will be fully parsed and checked (no matter what the source language had been), and the scoping rules will be those of `__import` instead of `#include`. This is a really big hammer, and I could imagine it smashing fingers if used poorly. I'm not sure this feature will pan out, but we need to try things to know. One big piece of that that I'll likely keep in either case is an overhaul of command-line options parsing for `slangc`. In particular, this logic has been moved into the core `slang` library (so that users can just pass options in via the API), and it is all done on UTF-8 strings rather than wide strings (which was always going to be Windows-specific). 19 June 2017, 16:56:42 UTC
cafed77 Tim Foley 16 June 2017, 17:05:42 UTC Merge pull request #28 from tfoleyNV/rewriter-operator-fix Bug fix: handle unchecked operator application in emit logic 16 June 2017, 17:05:42 UTC
9ba8ce3 Tim Foley 16 June 2017, 16:09:09 UTC Bug fix: handle unchecked operator application in emit logic When in rewriter mode, the emit logic will never see function applications inside function bodies, but it *will* see function application expressions at global scope, and some of these expressions might be unchecked. The challenge here is that even simple math operations now show up as function calls, so we need a bit of special-case logic to detect unchecked calls and then emit them using the syntax they were written with (e.g., use infix syntax if they were written as an infix expression). 16 June 2017, 16:09:09 UTC
0bf3d6d Tim Foley 16 June 2017, 00:18:00 UTC Merge pull request #27 from tfoleyNV/decl-ref-cleanup Replace `DeclRef` approach 16 June 2017, 00:18:00 UTC
e0389f5 Tim Foley 15 June 2017, 23:35:10 UTC Replace `DeclRef` approach For context: a `DeclRef` is supposed to capture both a pointer to a particualr declaration, and also any information needed to specialize that declaration for a context (e.g., generic parameter substitutions). The existing approach had a hiearchy of specialized decl-ref types that mirrored the AST hierarchy, but that led to a lot of boilerplate where you had to recapitulate the exact same hierarchy. The new appraoch basically treats `DeclRef<T>` as a sort of "smart pointer" in that it wraps a pointer to a `T` (the declaration), plus a side field for the specialization info, and then allows it to be cast as needed to other types (where the pointer cast would be allowed), while carrying along the side info. To enable this, all the things that used to be member functions of declaration-reference types are now free functions that take a `DeclRef<T>` for some specific `T` as a parameter. 15 June 2017, 23:35:10 UTC
1353a78 Tim Foley 15 June 2017, 22:54:35 UTC Merge pull request #26 from tfoleyNV/cleanup Cleanup 15 June 2017, 22:54:35 UTC
04d43cd Tim Foley 15 June 2017, 22:21:20 UTC Remove more "core" code that isn't used. It is always easier to add back code when you need it, than it is to maintain code you aren't using. 15 June 2017, 22:31:22 UTC
205187b Tim Foley 15 June 2017, 20:24:25 UTC Rename `CoreLib::*` to `Slang` Getting rid of more namespace complexity and stripping things down to the basics. This also gets rid of some dead code in the "core" library. 15 June 2017, 20:24:25 UTC
5175136 Tim Foley 15 June 2017, 20:12:51 UTC Rename `Slang::Compiler` -> `Slang` This gets rid of one unecessary namespace. 15 June 2017, 20:12:51 UTC
c34a433 Tim Foley 15 June 2017, 20:00:29 UTC Merge pull request #25 from tfoleyNV/interfaces Add basic support for `interface` declarations 15 June 2017, 20:00:29 UTC
367edf7 Tim Foley 15 June 2017, 18:16:10 UTC Add basic support for `interface` declarations - Add a test case for `interface` declarations and the exected implicit type conversion rules around them - Rename exising "trait" declaration kind to "interface" - There was already basic syntax for `__trait` declarations, and a bunch of related machinery. - Not all of it worked as needed, but it was clearly a start at solving the problem - Change `InterfaceConformanceDecl` to a more general `InheritanceDecl` that covers inheritance from any type expression (leave it to other code to validate the cases that should be allowed) - Instead of keeping a raw `bases` array on interface/trait declarations, turn all inheritance clauses into `IheritanceDecl` members - Add support for inheritance clause on `struct` types - Remove the `__conforms` syntax only used in the stdlib, in favor of conentional `: Base` style syntax already in place for aggregate types - Make sure that the parser pushes a new scope around he member declarations of an aggregate type, so that lookup in member functions will correctly find members of the enclosing type - In `TryCoerceImpl`, allow a type that conforms to an interface to be implicitly conveted to the corresponding interface type. This leaves out a lot of major functionality: - There is no validation that a type provides all the members it is supposed to as part of fulfilling a claimed interface conformance - The lookup process needs to deal with inherited members at some point. - We can avoid this for now if we don't allow inheritance for concrete types - When it comes time to handle it, it *might* be possible to implement by considering an `InheritanceDecl` to be, conceptually, a member of the inherited type, with a `__transparent` modifier - The lookup rules member functions do *not* deal with a lot of stuff: - There is no `this` expression right now - The semantic checker does not rewrite `foo` to `this.foo`, so downstream stages aren't going to get things in a clean format - There is no handling of mutability currently - The right answer there is probably to make member functions on `struct` types non-mutating by default, and add a qualifier to opt in to mutability. I believe this is actually what the OOP syntax in HLSL did way back when. - There is no handling of `static` members, and thus no checking to make sure that non-static members aren't referenced in static functions - None of this affects down-stream code generation right now, so it probably won't actually produce anything valid. - This is where we start needing a suitable IR to use for lowering, to manage the complexity. 15 June 2017, 19:42:52 UTC
3491d35 Tim Foley 14 June 2017, 22:10:05 UTC Merge pull request #7 from tfoleyNV/testing AppVeyor: Have AppVeyor run our tests. 14 June 2017, 22:10:05 UTC
90d6a40 Tim Foley 14 June 2017, 16:10:22 UTC AppVeyor: Run tests as part of AppVeyor builds This includes a bunch of related changes: - `slang-test` - Add a notion of an "output mode" that specifies whether we output to console (the default), or invoke the apprpriate AppVeyor command to update test status - Add a notion of test categories, so that tests can be tagged with categories, and then we can invoke only those tets in a given category, or choose to *exclude* tests with specific categories - Allow the `OSProcessSpawner` to look up an executable by "path" (meaning a full path is expected) or by "name" (meaning it should be allowed to look in the current directory, `PATH` environment variable, etc.). This was important to make sure that I can run `appveyor` without having to know its absolute path. - AppVeyor configuration - Change badge to reflect new build account for organization (rather than a single-user account) - Remove attempt to set AppVeyor build version in a clever way, since it breaks links from GitHub to AppVeyor - Change order or configurations in the build matrix to front-load the Release build (which has the main tests) - Turn on `fast_finish` flag so we don't have to wait as long for failed builds - Turn on `parallel` builds - Set `verbosity: minimal` to avoid getting build spew about Xamarin stuff I'm not using - Add custom `test_script` to invoke `test.bat` - Sets the test category based on teh build configuration, so we don't run the full test suite on every input. - `test.bat` - Allow for `-platform` and `-configuration` arguments - Rewrute a platform of `Win32` over to `x86` to match how the output directories are named - Futz around with how the directories are being passed along to work around annoying `.bat` file quoting behavior (I still don't get how batch files work) - Tests - Mark a bunch of tests as `smoke` tests - Mark the relevant tests as `render` tests (these get filtered out for AppVeyor builds) 14 June 2017, 21:40:48 UTC
8ddf03f Tim Foley 14 June 2017, 16:03:06 UTC Merge pull request #6 from tfoleyNV/testing Testing improvements 14 June 2017, 16:03:06 UTC
34c7cdc Tim Foley 14 June 2017, 15:35:43 UTC Testing: Adding binding-generation tests for some GLSL shaders Most of the Vulkan GLSL shaders in the `sascha-willems` corpus use completely regular parameter bindings, e.g.: ``` layout (binding = 0) uniform sampler2D samplerPositionDepth; layout (binding = 1) uniform sampler2D samplerNormal; layout (binding = 2) uniform sampler2D ssaoNoise; ``` With the Slang compiler, we can write this kind of stuff more compactly as: ``` uniform sampler2D samplerPositionDepth; uniform sampler2D samplerNormal; uniform sampler2D ssaoNoise; ``` and get the exact same result (in fact, we will generate output GLSL matching the first example). There are a few spot tests for this in the HLSL case, but I hadn't done anything for GLSL yet. Now that we have the ability to specify multiple tests to run on each input file, it was easy enough to go in and tweak some of these files to be usable as binding-generation tests. I didn't ammend all of the GLSL shaders for two reasons: 1. Not all of the shaders will work with completely automatic binding generation done on a per-file basis. In some cases, the parameter layout needs to consider multiple files (and the order in which the files are supplied could matter). This is probably best handled with more directed tests. 2. There is going to be a ton of duplication if I just have 100s of tests that all confirm that, yes, the Slang compiler can count vertex inputs starting from zero. These shaders aren't really presenting a whole lot of unique cases to work with. That said, a level of confidence greater than zero is always better than zero confidence. 14 June 2017, 15:35:43 UTC
604e0fc Tim Foley 14 June 2017, 15:23:49 UTC Testing: Add support for multiple tests per input file. Now if there are multiple tests listed for an input file like `foo.slang`, they will be output as: passed test: 'foo.slang' passed test: 'foo.slang.1' passed test: 'foo.slang.2' This isn't a perfect solution, but it should work well enough for now. This change doesn't make any tests actually use the new capability. 14 June 2017, 15:23:49 UTC
5a11b47 Tim Foley 13 June 2017, 21:10:10 UTC Merge pull request #5 from tfoleyNV/cross-compile Cross compilation 13 June 2017, 21:10:10 UTC
d81c347 Tim Foley 12 June 2017, 19:26:12 UTC First pass at support for cross-compilation This is a large change that contains many pieces: - Update the `cross-compile0` test to actually make use of cross compilation. Now the `cross-compile0.hlsl` file contains both HLSL and GLSL source code, and then imports code from `cross-compile0.slang`, which provides a "library" (one function) that can be shared between both the HLSL and GLSL version of things. - Fixed a bug in the support for backslash-escaped newlines. - Added a new `__import` declaration type (replaces the `using` directive that was still around in a vestigial form) An `__import` causes the compiler to look for a Slang source file (currently using the ordinary `#include` lookup logic), and then parse/check the found file as an additional module ("translation unit"), before making its declarations visible in the current scope. - Refactored the main compilation flow to be simpler. There were the `ShaderCompiler` and `ShaderCompilerImpl` classes that weren't relaly doing anything, but added complexity to the whole workflow. - The `render-test` application has been heavily modified to better support testing cross-compilation workflows. At the most basic level we are starting to distinguish pass-through vs. rewriter workflows, and are passing various `#define`s down to the compiler(s) to let the source code be customized as needed for each case. Several annoying corner cases are caused here by having to support the GLSL compilation model, which really wants each entry point in its own specific translation unit, whereas we really want to keep things nicely contained in single files. - Added support for `__intrinsic` operations to have target-specific behavior. This allows a function to be given a different name for some specific target (so a call gets emitted as a call to that other operation). More generally, the library writer can put together an arbitrary format string that will be used in place of expressions that call the given function, e.g.: __intrinsic(hlsl, "$1 - $0") __intrinsic int foo(int a, int b); Given this declaration, a call like `foo(x,y)` will code generate as `x - y` for HLSL, and as `foo(x,y)` for all other targets. Annoying things still to be dealt with: - The way that I'm filtering the user-provided options when passing things down to the compilation of dynamically loaded modules is a bit ad hoc. It would be good to have a systematic notion of which options will be inherited and which won't. There is also more code duplication than I'd like, so we risk having the compiler behave differently when compiling a file at the top level, vs. because of `__import`. - Adding target-specific behavior to intrinsics is all well and good, but the current approach means we can only add this to the original declaration, which limits the ability to easily extend the set of targets. A better approach long-term would be to add a more robust notion of target-based overload resolution (which would happen after semantic checking). Then one mechanism would be used to find the right target-specific overload to use for an operation, and then each (target-specific) definition could use a simpler attribute to intercept code-generation behavior. Note that we might eventually need a similar notion to deal with stage- or profile-specific functions and the overloading behavior around them, so using this for intrinsics doesn't seem like a bad idea. 13 June 2017, 20:53:47 UTC
a14f3c5 Tim Foley 12 June 2017, 22:16:32 UTC Fixups for newline-escaping behavior. This is really messy and I'm not entirely happy with the result, but at least we handle a couple more corner cases. 13 June 2017, 19:25:29 UTC
7fc4c40 Tim Foley 12 June 2017, 22:34:12 UTC Merge pull request #4 from tfoleyNV/escaped-newlines Escaped newlines 12 June 2017, 22:34:12 UTC
97fc943 Tim Foley 12 June 2017, 19:37:07 UTC Add test case for escaped newlines. This also serves as a regression test for the recent preprocessor bug fix. 12 June 2017, 21:33:52 UTC
e95b631 Tim Foley 12 June 2017, 21:28:00 UTC Lexer: handle escaped newlines This is mostly to allow for the idiomatic style of defining a multi-line macro in C: #define FOO(a,b) \ x(a) \ y(b) \ /* end */ The handling is reasonably general: in the lexer whenever we need to consume or "peek" the next code point, we check if we are at the start of a backslash-newline sequence, and if so we skip past that to find what we were looking for. However, the way I'm handling things right now there is no step taken to "clean" a token and remove the backslash or newline from its value, so downstream code that actually inspects token values will probably break if users start putting escaped newlines in the middle of names. We can fix that issue when (if) it comes up. 12 June 2017, 21:33:52 UTC
b53041b Tim Foley 12 June 2017, 21:20:50 UTC Preprocessor: fix bug with multi-argument macros. There was a subtle bug when a function-like macro with multiple arguments expands to use the arguments one after the other: #define FOO(a,b) a b FOO(int, x); During expansion, the input streams look something like (using `.` to represent the cursor): // macro invocation: FOO(int, x) . ; // macro expansion of `FOO(int, x)` a . b // macro argument `a` int . That is, we are at the end of the first argument's tokens. When "peeking" the next token, we correctly work up the list of active streams until we find one that isn't at its end, and that gives us the token `b`. But then we need to look up `b` in an appropriate environment to find what it is bound to. Each of the streams above has an environment asociated with it, and in particular, `b` is only defined in the middle environment, because that is where the macro arguments were registered. The simple fix here is to make the lookup logic for finding an environment follow the same logic as finding the next token. A more complete fix down the line could involve getting rid of the approach of allowing an input stream to be "active" but at its end. I believe this was originally required to handle some error cases in directives, where we'd want to keep the input stream for one file active until we are done parsing a full directive from it (e.g., if a directive is on the last line of the file). Now that we generate an explicit "end of directive" token, that may not be required. 12 June 2017, 21:33:52 UTC
ce90fec Tim Foley 12 June 2017, 21:33:31 UTC Merge pull request #3 from tfoleyNV/rename-tests Rename tests from `*.spire` to `*.slang` 12 June 2017, 21:33:31 UTC
5b49974 Tim Foley 12 June 2017, 19:42:17 UTC Rename tests from `*.spire` to `*.slang` Many of the existing test cases were being skipped on accident, because their file names used `.spire` and the test tool was now looking for `.slang` 12 June 2017, 20:00:47 UTC
d1f11a2 Tim Foley 12 June 2017, 19:28:12 UTC Merge pull request #2 from tfoleyNV/glsl-render-test GLSL: get GLSL limping in `render-test` 12 June 2017, 19:28:12 UTC
5fe2cf3 Tim Foley 12 June 2017, 17:48:36 UTC GLSL: get GLSL limping in `render-test` The test case that is there right now is nominally a cross-compilation test, but for right now it uses the preprocessor to present completely different code for HLSL and GLSL compilation. This change is really just fleshing out the OpenGL side of `render-test` enough that it can produce images using OpenGL to enable further testing. 12 June 2017, 19:12:20 UTC
4d63b6f Tim Foley 12 June 2017, 16:08:15 UTC Merge pull request #1 from tfoleyNV/appveyor-build-version AppVeyor: try to produce a better build version 12 June 2017, 16:08:15 UTC
5dc1586 Tim Foley 12 June 2017, 15:54:32 UTC AppVeyor: PowerShell requires `elseif` not `else if` 12 June 2017, 15:54:32 UTC
d93c210 Tim Foley 12 June 2017, 15:49:15 UTC AppVeyor: Try to fix errors in `appveyor.yml` - Also add link to build "badge" so that I can more easily watch build status. 12 June 2017, 15:50:21 UTC
62687cd Tim Foley 12 June 2017, 15:38:31 UTC AppVeyor: try to produce a better build version I don't want to have to try and manually keep a version number in `appveyor.yml` up to date with any versioning for releases, etc. This change tries to derive a reasonable version name from a Git tag, a pull request number, or a branch name (in that order). I then append the AppVeyor build number to the end to try to ensure that we always have something unique (which is a requirement for AppVeyor). 12 June 2017, 15:38:31 UTC
f56fba2 Tim Foley 10 June 2017, 00:30:56 UTC AppVeyor: Try to clone submodules, and add badge - Add an `install` step that tries to update our submodules (just `glslang` at the moment) - Try to add a "badge" image to the main `README.md` 10 June 2017, 00:30:56 UTC
2495f6b Tim Foley 10 June 2017, 00:24:23 UTC appveyor: Add initial appveyor.yml 10 June 2017, 00:24:23 UTC
383bc76 Tim Foley 10 June 2017, 00:18:39 UTC Build: more fixes to get `msbuild` to work from command line. All of this is just related to cruft left over from the old project setup. 10 June 2017, 00:18:39 UTC
44e3f61 Tim Foley 09 June 2017, 23:00:00 UTC Fix: Remove some old project references. There were some dead project references lying around after the Spire->Slang rename, that don't affect builds from inside Visual Studio, but seem to break stand-alone build with `msbuild`. 10 June 2017, 00:10:47 UTC
f7c10df Tim Foley 09 June 2017, 20:54:17 UTC glslang: Fixups for build Some files seem to have been removed since I last sync'ed the glslang code. 09 June 2017, 20:54:17 UTC
8706f17 Tim Foley 09 June 2017, 20:50:36 UTC Add submodule for `glslang` This is currently required for GLSL->SPIR-V compilation and related tests. 09 June 2017, 20:50:36 UTC
fcf83db Tim Foley 09 June 2017, 18:34:21 UTC Initial import of code. 09 June 2017, 20:44:59 UTC
52e8d4b Tim Foley 09 June 2017, 17:42:49 UTC Initial commit 09 June 2017, 17:42:49 UTC
back to top