Skip to content

Commit 504ffac

Browse files
committed
Copy proposals from the Slang repo
This is a copy of docs/proposal from the Slang repository [1] at revision [2]. [1] https://github.com/shader-slang/slang [2] 79aebc18d54db3f0be8bd6529c0d79f4d8d4fc58 This helps to address shader-slang/slang#6155.
1 parent 3b6e0d9 commit 504ffac

28 files changed

+8299
-0
lines changed

proposals/000-template.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
SP #000: Proposal Template
2+
=================
3+
4+
This document provides a starting point for a larger feature proposal.
5+
The sections in it are suggested, but can be removed if they don't make sense for a chosen feature.
6+
7+
The first section should provide a concise description of **what** the feature is and, if possible, **why** it is important.
8+
9+
A proposal for a Slang language/compiler feature or system should start with a concise description of what the feature it and why it could be important.
10+
11+
Status
12+
------
13+
14+
Status: Design Review/Planned/Implementation In-Progress/Implemented/Partially Implemented. Note here whether the proposal is unimplemented, in-progress, has landed, etc.
15+
16+
Implementation: [PR 000] [PR 001] ... (list links to PRs)
17+
18+
Author: authors of the design doc and the implementation.
19+
20+
Reviewer: Reviewers of the proposal and implementation.
21+
22+
Background
23+
----------
24+
25+
The background section should explain where things stand in the language/compiler today, along with any relevant concepts or terms of art from the wider industry.
26+
If the proposal is about solving a problem, this section should clearly illustrate the problem.
27+
If the proposal is about improving a design, it should explain where the current design falls short.
28+
29+
Related Work
30+
------------
31+
32+
The related work section should show examples of how other languages, compilers, etc. have solved the same or related problems. Even if there are no direct precedents for what is being proposed, there should ideally be some points of comparison for where ideas sprang from.
33+
34+
Proposed Approach
35+
-----------------
36+
37+
Explain the idea in enough detail that a reader can concretely know what you are proposing to do. Anybody who is just going to *use* the resulting feature/system should be able to read this and get an accurate idea of what that experience will be like.
38+
39+
Detailed Explanation
40+
--------------------
41+
42+
Here's where you go into the messy details related to language semantics, implementation, corner cases and gotchas, etc.
43+
Ideally this section provides enough detail that a contributor who wasn't involved in the proposal process could implement the feature in a way that is faithful to the original.
44+
45+
Alternatives Considered
46+
-----------------------
47+
48+
Any important alternative designs should be listed here.
49+
If somebody comes along and says "that proposal is neat, but you should just do X" you want to be able to show that X was considered, and give enough context on why we made the decision we did.
50+
This section doesn't need to be defensive, or focus on which of various options is "best".
51+
Ideally we can acknowledge that different designs are suited for different circumstances/constraints.

proposals/001-where-clauses.md

Lines changed: 348 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
Allow Type Equality Constraints on Generics
2+
===========================================
3+
4+
We propose to allow *type equality* constraints in `where` clauses.
5+
6+
Status
7+
------
8+
9+
In progress.
10+
11+
Background
12+
----------
13+
14+
As of proposal [001](001-where-clauses.md), Slang allows for generic declarations to include a *`where` clause* which enumerates constraints on the generic parameters that must be satisfied by any arguments provided to that generic:
15+
16+
V findOrDefault<K, V>( HashTable<K,V> table, K key )
17+
where K : IHashable,
18+
V : IDefaultInitializable
19+
{ ... }
20+
21+
Currently, the language only accepts *conformance* constraints of the form `T : IFoo`, where `T` is one of the parameters of the generic, and `IFoo` is either an `interface` or a conjunction of interfaces, which indicate that the type `T` must conform to `IFoo`.
22+
23+
This proposal is motivated by the observation that when an interface has associated types, there is currently no way for a programmer to introduce a generic that is only applicable when an associated type satisfies certain constraints.
24+
25+
As an example, consider an interface for types that can be "packed" into a smaller representation for in-memory storage (instead of a default representation optimized for access from registers):
26+
27+
interface IPackable
28+
{
29+
associatedtype Packed;
30+
31+
init(Packed packed);
32+
Packed pack();
33+
}
34+
35+
Next, consider an hypothetical interface for types that can be deserialized from a stream:
36+
37+
interface IDeserializable
38+
{
39+
init( InputStream stream );
40+
}
41+
42+
Given these definitions, we might want to define a function that takes a packable type, and deserializes it from a stream:
43+
44+
T deserializePackable<T>( InputStream stream )
45+
where T : IPackable
46+
{
47+
return T( T.Packed(stream) );
48+
}
49+
50+
As written, this function will fail to compile because the compiler cannot assume that `T.Packed` conforms to `IDeserializable`, in order to support initialization from a stream.
51+
52+
A brute-force solution would be to add the `IDeserializable` constraint to the `IPackable.Packed` associated type, but doing so may not be consistent with the vision the designer of `IPackable` had in mind. Indeed, there is no reason to assume that `IPackable` and `IDeserializable` even have the same author, or are things that the programmer trying to write `deserializePackable` can change.
53+
54+
It might seem that we could improve the situation by introducing another generic type parameter, so that we can explicitly constraint it to be deserializable:
55+
56+
T deserializePackable<T, U>( InputStream stream )
57+
where T : IPackable,
58+
P : IDeserializable
59+
{
60+
return T( U(stream) );
61+
}
62+
63+
This second attempt *also* fails to compile.
64+
In this case, there is no way for the compiler to know that `T` can be initialized from a `P`, because it cannot intuit that `P` is meant to be `T.Packed`.
65+
66+
Our two failed attempts can each be fixed by introducing two new kinds of constraints:
67+
68+
* Conformance constraints on associated types: `T.A : IFoo`
69+
70+
* Equality constraints on associated types: `T.A == X`
71+
72+
Related Work
73+
------------
74+
75+
Both Rust and Swift support additional kinds of constraints on generics, including the cases proposed here.
76+
The syntax in those languages matches what we propose.
77+
78+
Proposed Approach
79+
-----------------
80+
81+
In addition to conformance constraints on generic type parameters (`T : IFoo`), the compiler will also support constraints on associated types of those parameters (`T.A : IFoo`), and associated types of those associated types (`T.A.B : IFoo`), etc.
82+
83+
In addition, the compiler will accept constraints that restrict an associated type (`T.A`, `T.A.B`, etc.) to be equal to some other type.
84+
The other type may be a concrete type, another generic parameter, or another associated type.
85+
86+
Detailed Explanation
87+
--------------------
88+
89+
### Parser
90+
91+
The parser already supports nearly arbitrary type exprssions on both sides of a conformance constraint, and then validates that the types used are allowed during semantic checking.
92+
The only change needed at that level is to split `GenericTypeConstraintDecl` into two cases: one for conformance constraints, and another for equality constraints, and then to support constraints with `==` instead of `:`.
93+
94+
### Semantic Checking
95+
96+
During semantic checking, instead of checking that the left-hand type in a constraint is always one of the generic type parameters, we could instead check that the left-hand type expression is either a generic type parameter or `X.AssociatedType` where `X` would be a valid left-hand type.
97+
98+
The right-hand type for conformance constraints should be checked the same as before.
99+
100+
The right-hand type for an equality constraint should be allowed to be an arbitrary type expression that names a proper (and non-`interface`) type.
101+
102+
One subtlety is that in a type expression like `T.A.B` where both `A` and `B` are associated types, it may be that the `B` member of `T.A` can only be looked up because of another constraint like `T.A : IFoo`.
103+
When performing semantic checking of a constraint in a `where` clause, we need to decide which of the constraints may inform lookup when resolving a type expression like `X.A`.
104+
Some options are:
105+
106+
* We could consider only constraints that appear before the constraint that includes that type expression. In this case, a programmer must always introduce a constraint `X : IFoo` before a constraint that names `X.A`, if `A` is an associated type introduced by `IFoo`.
107+
108+
* We could consider *all* of the constraints simultaneously (except, perhaps, the constraint that we are in the middle of checking).
109+
110+
The latter option is more flexible, but may be (much) harder to implement in practice.
111+
We propose that for now we use for first option, but remain open to implementing the more general case in the future.
112+
113+
Given an equality constraint like `T.A.B == X`, semantic checking needs detect cases where an `X` is used and a `T.A.B` is expected, or vice versa.
114+
These cases should introduce some kind of cast-like expression, which references the type equality witness as evidence that the cast is valid (and should, in theory, be a no-op).
115+
116+
Semantic checking of equality constraints should identify contradictory sets of constraints.
117+
Such contradictions can be simple to spot:
118+
119+
interface IThing { associatedtype A; }
120+
void f<T>()
121+
where T : IThing,
122+
T.A == String,
123+
T.A == Float,
124+
{ ... }
125+
126+
but they can also be more complicated:
127+
128+
void f<T,U>()
129+
where T : IThing,
130+
U : IThing,
131+
T.A == String,
132+
U.A == Float,
133+
T.A == U.A
134+
{ ... }
135+
136+
In each case, an associated type is being constrained to be equal to two *different* concrete types.
137+
The is no possible set of generic arguments that could satisfy these constraints, so declarations like these should be rejected.
138+
139+
We propose that the simplest way to identify and diagnose contradictory constraints like this is during canonicalization, as described below.
140+
141+
### IR
142+
143+
At the IR level, a conformance constraint on an associated type is no different than any other conformance constraint: it lowers to an explicit generic parameter that will accept a witness table as an argument.
144+
145+
The choice of how to represent equality constraints is more subtle.
146+
One option is to lower an equality constraint to *nothing* at the IR level, under the assumption that the casts that reference these constraints should lower to nothing.
147+
Doing so would introduce yet another case where the IR we generate doesn't "type-check."
148+
The other option is to lower a type equality constraint to an explicit generic parameter which is then applied via an explicit op to convert between the associated type and its known concrete equivalent.
149+
The representation of the witnesses required to provide *arguments* for such parameters is something that hasn't been fully explored, so for now we propose to take the first (easier) option.
150+
151+
### Canonicalization
152+
153+
Adding new kinds of constraints affects *canonicalization*, which was discussed in proposal 0001.
154+
Conformane constraints involving associated types should already be order-able according to the rules in that proposal, so we primarily need to concern ourselves with equality constraints.
155+
156+
We propose the following approach:
157+
158+
* Take all of the equality constraints that arise after any expansion steps
159+
* Divide the types named on either side of any equality constraint into *equivalence classes*, where if `X == Y` is a constraint, then `X` and `Y` must in the same equivalence class
160+
* Each type in an equivalence class will either be an associated type of the form `T.A.B...Z`, derived from a generic type parameter, or a *independent* type, which here means anything other than those associated types.
161+
* Because of the rules enforced during semantic checking, each equivalence class must have at least one associated type in it.
162+
* Each equivalence class may have zero or more independent types in it.
163+
* For each equivalence class with more than one independent type in it, diagnose an error; the application is attempting to constrain one or more associated types to be equal to multiple distinct types at once
164+
* For each equivalence class with exactly one independent type in it, produce new constraints of the form `T.A.B...Z == C`, one for each associated type in the equivalence class, where `C` is the independent type
165+
* For each equivalence class with zero independent types in it, pick the *minimal* associated type (according to the type ordering), and produce new constraints of the form `T.A... == U.B...` for each *other* associated type in the equivalence class, where `U.B...` is the minimal associated type.
166+
* Sort the new constraints by the associated type on their left-hand side.
167+
168+
Alternatives Considered
169+
-----------------------
170+
171+
The main alternative here would be to simply not have these kinds of constraints, and push programmers to use type parameters instead of associated types in cases where they want to be able to enforce constraints on those types.
172+
E.g., the `IPackable` interface from earlier could be rewritten into this form:
173+
174+
175+
interface IPackable<Packed>
176+
{
177+
init(Packed packed);
178+
Packed pack();
179+
}
180+
181+
With this form for `IPackable`, it becomes possible to use additional type parameters to constraint the `Packed` type:
182+
183+
T deserializePackable<T, U>( InputStream stream )
184+
where T : IPackable<U>,
185+
P : IDeserializable
186+
{
187+
return T( U(stream) );
188+
}
189+
190+
While this workaround may seem reasomable in an isolated example like this, there is a strong reason why languages like Slang choose to have both generic type parameters (which act as *inputs* to an abstraction) and associated types (which act as *outputs*).
191+
We believe that associated types are an important feature, and that they justify the complexity of these new kinds of constraints.

proposals/003-atomic-t.md

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
SP #003 - `Atomic<T>` type
2+
==============
3+
4+
5+
Status
6+
------
7+
8+
Author: Yong He
9+
10+
Status: Implemented.
11+
12+
Implementation: [PR 5125](https://github.com/shader-slang/slang/pull/5125)
13+
14+
Reviewed by: Theresa Foley, Jay Kwak
15+
16+
Background
17+
----------
18+
19+
HLSL defines atomic intrinsics to work on free references to ordinary values such as `int` and `float`. However, this doesn't translate well to Metal and WebGPU,
20+
which defines `atomic<T>` type and only allow atomic operations to be applied on values of `atomic<T>` types.
21+
22+
Slang's Metal backend follows the same technique in SPIRV-Cross and DXIL->Metal converter that relies on a C++ undefined behavior that casts an ordinary `int*` pointer to a `atomic<int>*` pointer
23+
and then call atomic intrinsic on the reinterpreted pointer. This is fragile and not guaranteed to work in the future.
24+
25+
To make the situation worse, WebGPU bans all possible ways to cast a normal pointer into an `atomic` pointer. In order to provide a truly portable way to define
26+
atomic operations and allow them to be translatable to all targets, we will also need an `atomic<T>` type in Slang that maps to `atomic<T>` in WGSL and Metal, and maps to
27+
`T` for HLSL/SPIRV.
28+
29+
30+
Proposed Approach
31+
-----------------
32+
33+
We define an `Atomic<T>` type that functions as a wrapper of `T` and provides atomic operations:
34+
```csharp
35+
enum MemoryOrder
36+
{
37+
Relaxed = 0,
38+
Acquire = 1,
39+
Release = 2,
40+
AcquireRelease = 3,
41+
SeqCst = 4,
42+
}
43+
44+
[sealed] interface IAtomicable {}
45+
[sealed] interface IArithmeticAtomicable : IAtomicable, IArithmetic {}
46+
[sealed] interface IBitAtomicable : IArithmeticAtomicable, IInteger {}
47+
48+
[require(cuda_glsl_hlsl_metal_spirv_wgsl)]
49+
struct Atomic<T : IAtomicable>
50+
{
51+
T load(MemoryOrder order = MemoryOrder.Relaxed);
52+
53+
[__ref] void store(T newValue, MemoryOrder order = MemoryOrder.Relaxed);
54+
55+
[__ref] T exchange(T newValue, MemoryOrder order = MemoryOrder.Relaxed); // returns old value
56+
57+
[__ref] T compareExchange(
58+
T compareValue,
59+
T newValue,
60+
MemoryOrder successOrder = MemoryOrder.Relaxed,
61+
MemoryOrder failOrder = MemoryOrder.Relaxed);
62+
}
63+
64+
extension<T : IArithmeticAtomicable> Atomic<T>
65+
{
66+
[__ref] T add(T value, MemoryOrder order = MemoryOrder.Relaxed); // returns original value
67+
[__ref] T sub(T value, MemoryOrder order = MemoryOrder.Relaxed); // returns original value
68+
[__ref] T max(T value, MemoryOrder order = MemoryOrder.Relaxed); // returns original value
69+
[__ref] T min(T value, MemoryOrder order = MemoryOrder.Relaxed); // returns original value
70+
}
71+
72+
extension<T : IBitAtomicable> Atomic<T>
73+
{
74+
[__ref] T and(T value, MemoryOrder order = MemoryOrder.Relaxed); // returns original value
75+
[__ref] T or(T value, MemoryOrder order = MemoryOrder.Relaxed); // returns original value
76+
[__ref] T xor(T value, MemoryOrder order = MemoryOrder.Relaxed); // returns original value
77+
[__ref] T increment(MemoryOrder order = MemoryOrder.Relaxed); // returns original value
78+
[__ref] T decrement(MemoryOrder order = MemoryOrder.Relaxed); // returns original value
79+
}
80+
81+
extension int : IArithmeticAtomicable {}
82+
extension uint : IArithmeticAtomicable {}
83+
extension int64_t : IBitAtomicable {}
84+
extension uint64_t : IBitAtomicable {}
85+
extension double : IArithmeticAtomicable {}
86+
extension float : IArithmeticAtomicable {}
87+
extension half : IArithmeticAtomicable {}
88+
89+
// Operator overloads:
90+
// All operator overloads are using MemoryOrder.Relaxed semantics.
91+
__prefix T operator++<T>(__ref Atomic<T> v); // returns new value.
92+
__postfix T operator++<T>(__ref Atomic<T> v); // returns original value.
93+
__prefix T operator--<T>(__ref Atomic<T> v); // returns new value.
94+
__postfix T operator--<T>(__ref Atomic<T> v); // returns original value.
95+
T operator+=(__ref Atomic<T> v, T operand); // returns new value.
96+
T operator-=(__ref Atomic<T> v, T operand); // returns new value.
97+
T operator|=(__ref Atomic<T> v, T operand); // returns new value.
98+
T operator&=(__ref Atomic<T> v, T operand); // returns new value.
99+
T operator^=(__ref Atomic<T> v, T operand); // returns new value.
100+
```
101+
102+
We allow `Atomic<T>` to be defined in struct fields, as array elements, as elements of `RWStructuredBuffer` types,
103+
or as groupshared variable types or `__ref` function parameter types. For example:
104+
105+
```hlsl
106+
struct MyType
107+
{
108+
int ordinaryValue;
109+
Atomic<int> atomicValue;
110+
}
111+
112+
RWStructuredBuffer<MyType> atomicBuffer;
113+
114+
void main()
115+
{
116+
atomicBuffer[0].atomicValue.atomicAdd(1);
117+
printf("%d", atomicBuffer[0].atomicValue.load());
118+
}
119+
```
120+
121+
In groupshared memory:
122+
123+
```hlsl
124+
void main()
125+
{
126+
groupshared atomic<int> c;
127+
c.atomicAdd(1);
128+
}
129+
```
130+
131+
Note that in many targets, it is invalid to use `atomic<T>` type to define a local variable or a function parameter, or in any way
132+
to cause a `atomic<T>` to reside in local/function/private address space. Slang should be able to lower the type
133+
into its underlying type. The use of atomic type in these positions will simply have no meaning. However, we are going to leave
134+
this legalization as future work and leave such situation as undefined behavior for now.
135+
136+
This should be handled by a legalization pass similar to `lowerBufferElementTypeToStorageType` but operates
137+
in the opposite direction: the "loaded" value from a buffer is converted into an atomic-free type, and storing a value leads to an
138+
atomic store at the corresponding locations.
139+
140+
For non-WGSL/Metal targets, we can simply lower the type out of existence into its underlying type.
141+
142+
# Related Work
143+
144+
`Atomic<T>` type exists in almost all CPU programming languages and is the proven way to express atomic operations over different
145+
architectures that have different memory models. WGSL and Metal follows this trend to require atomic operations being expressed
146+
this way. This proposal is to make Slang follow this trend and make `Atomic<T>` the recommended way to express atomic operation
147+
going forward.
148+
149+
# Future Work
150+
151+
As discussed in previous sections, we should consider adding a legalization pass to allow `Atomic<T>` type to be used anywhere in
152+
any memory space, and legalize them out to just normal types if they are used in memory spaces where atomic semantic has no/trivial
153+
meaning.

0 commit comments

Comments
 (0)