Skip to content

Commit 7061975

Browse files
tgodziktgodzik
authored
Improve quotes reflection reference documentation (scala#23385)
Not sure if this is enough, but the I would say most important overview (and most common pitfalls, from what I noticed in my experiences and from some submitted issues) are included. It's tough to see in the raw .md file here, but in Scaladoc there is a hierarchy shown, which I think helps with the readability: <img width="350" alt="Zrzut ekranu 2025-06-17 o 20 56 25" src="https://github.com/user-attachments/assets/9a0fee74-a6d8-4ad5-acb4-f2180e82617d" /> I think another improvement here could be done by specifying in https://scala-lang.org/api/3.3_LTS/scala/quoted/Quotes$reflectModule.html# which AST trees correspond to what code, but that's probably a little tricky to do cleanly
2 parents fee3891 + 0b335cc commit 7061975

File tree

1 file changed

+194
-16
lines changed

1 file changed

+194
-16
lines changed

‎docs/_docs/reference/metaprogramming/reflection.md

Lines changed: 194 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,17 @@ nightlyOf: https://docs.scala-lang.org/scala3/reference/metaprogramming/reflecti
55
---
66

77
Reflection enables inspection and construction of Typed Abstract Syntax Trees
8-
(Typed-AST). It may be used on quoted expressions (`quoted.Expr`) and quoted
9-
types (`quoted.Type`) from [Macros](./macros.md) or on full TASTy files.
8+
(Typed-AST).
109

10+
It may be used on quoted expressions (`quoted.Expr`) and quoted
11+
types (`quoted.Type`) from [Macros](./macros.md) or [multi-staging-programming](./staging.md),
12+
or on whole TASTy files (via [tasty-inspection](./tasty-inspect.md)).
1113
If you are writing macros, please first read [Macros](./macros.md).
1214
You may find all you need without using quote reflection.
1315

14-
## API: From quotes and splices to TASTy reflect trees and back
16+
## Converting `Expr`s to TASTy reflect trees and back
1517

16-
With `quoted.Expr` and `quoted.Type` we can compute code but also analyze code
18+
With `quoted.Expr` and `quoted.Type` we can not only compute code but also analyze code
1719
by inspecting the ASTs. [Macros](./macros.md) provide the guarantee that the
1820
generation of code will be type-correct. Using quote reflection will break these
1921
guarantees and may fail at macro expansion time, hence additional explicit
@@ -33,10 +35,79 @@ def natConstImpl(x: Expr[Int])(using Quotes): Expr[Int] =
3335
...
3436
```
3537

36-
### Extractors
38+
We can access the underlying typed AST of an `Expr` using the `asTerm` extension method:
3739

38-
`import quotes.reflect.*` will provide all extractors and methods on `quotes.reflect.Tree`s.
39-
For example the `Literal(_)` extractor used below.
40+
```scala
41+
val term: Term = x.asTerm
42+
```
43+
44+
Similarly, you can change a `Term` back into an `Expr` with `.asExpr` (returning `Expr[Any]`)
45+
or `.asExprOf[T]` (returning `Expr[T]`, with an exception being thrown at macro-expansion time if the type does not conform).
46+
47+
## Constructing and Analysing trees
48+
49+
Generally, there are 3 main types of constructs you need to know to properly construct and analyse Typed ASTs:
50+
* Trees
51+
* Symbols with Flags
52+
* TypeReprs
53+
54+
### Typed Abstract Syntax Trees
55+
Typed AST is a tree-like representation of the code of a program achieved after typing.
56+
It’s represented by the `Tree` type in the reflection API.
57+
58+
`Terms` are subtypes of trees that represent an expression of certain value. Because of this,
59+
they always have a type associated with them (accessible with `.tpe`). `Terms` can be transformed into `Exprs` with `.asExpr`.
60+
61+
Let’s look at an example in how the `Trees` map into real scala code:
62+
63+
```scala
64+
val foo: Int = 0
65+
```
66+
The above is represented in the quotes reflect API by a `ValDef` (a subtype of `Tree`, but not `Term`!):
67+
```scala
68+
ValDef(foo,Ident(Int),Literal(Constant(0))) // ValDef is a subtype of Tree but not Term
69+
```
70+
71+
```scala
72+
val foo: Int = 0
73+
foo + 1
74+
```
75+
The above is represented in the quotes reflect API by a `Block` (a subtype of `Term`, itself a subtype of `Tree`)
76+
```scala
77+
Block(
78+
List(
79+
ValDef(foo,Ident(Int),Literal(Constant(0)))
80+
),
81+
Apply(
82+
Select(Ident(foo),+),
83+
List(Literal(Constant(1)))
84+
)
85+
)
86+
```
87+
88+
You can see the whole hierarchy between different types of Trees in
89+
[`reflectModule` documentation](https://scala-lang.org/api/3.3_LTS/scala/quoted/Quotes$reflectModule.html#`).
90+
91+
You can also check the shape of code by printing out quoted code transformed into a Term:
92+
```scala
93+
println( '{ scalaCode }.asTerm )
94+
```
95+
Bear in mind this will always produce a Term. E.g.:
96+
```scala
97+
'{
98+
val foo: Int = 0
99+
}.asTerm
100+
```
101+
Is represented as `Block(List(ValDef(foo,Ident(Int),Literal(Constant(0)))),Literal(Constant(())))`, which is actually a `Block` of `Unit` type:
102+
```scala
103+
'{
104+
val foo: Int = 0
105+
()
106+
}
107+
```
108+
#### Tree Extractors and Constructors
109+
`import quotes.reflect.*` provides all extractors, apply-based constructors and methods on `quotes.reflect.Tree`s.
110+
For example, see the `Literal(_)` extractor used below.
40111

41112
```scala
42113
def natConstImpl(x: Expr[Int])(using Quotes): Expr[Int] =
@@ -54,7 +125,7 @@ def natConstImpl(x: Expr[Int])(using Quotes): Expr[Int] =
54125
'{0}
55126
```
56127

57-
We can easily know which extractors are needed using `Printer.TreeStructure.show`,
128+
We can easily know which extractors/constructors are needed using `Printer.TreeStructure.show`,
58129
which returns the string representation the structure of the tree. Other printers
59130
can also be found in the `Printer` module.
60131

@@ -64,14 +135,121 @@ tree.show(using Printer.TreeStructure)
64135
Printer.TreeStructure.show(tree)
65136
```
66137

67-
The methods `quotes.reflect.Term.{asExpr, asExprOf}` provide a way to go back to
68-
a `quoted.Expr`. Note that `asExpr` returns a `Expr[Any]`. On the other hand
69-
`asExprOf[T]` returns a `Expr[T]`, if the type does not conform to it an exception
70-
will be thrown at runtime.
138+
Bear in mind that extractors and constructors for the same trees might be comprised of different arguments, e.g. for `ValDef` the `apply` method
139+
has `(Symbol, Option[Term])` arguments and `unapply` has `(String, TypeTree, Option[Term])` (if we want to obtain the symbol directly, we can call `.symbol` on the `ValDef`).
140+
141+
### Symbols
142+
To construct definition `Trees` we might have to create or use a `Symbol`. Symbols represent the "named" parts of the code, the declarations we can reference elsewhere later. Let’s try to create `val name: Int = 0` from scratch.
143+
To create a val like this, we need to first create a `Symbol` that matches the intended `Tree` type, so for a `ValDef` we would use the `Symbol.newVal` method:
144+
```scala
145+
import quotes.reflect._
146+
val fooSym = Symbol.newVal(
147+
parent = Symbol.spliceOwner,
148+
name = "foo",
149+
tpe = TypeRepr.of[Int],
150+
flags = Flags.EmptyFlags,
151+
privateWithin = Symbol.noSymbol
152+
)
153+
val tree = ValDef(fooSym, Some(Literal(IntConstant(0))))
154+
```
155+
Generally, every `Symbol` needs to have an parent/owner `Symbol`, signifying where it is defined.
156+
E.g if we want to define the val as part of a class, then naturally, we need that class' symbol to be the owner of the val symbol.
157+
You may also notice the flags and privateWithin arguments, which are explained later in the `Flags` chapter.
158+
159+
The created val can be later referenced in other parts of the generated code with the use of `Ref` (a subtype of `Term`):
160+
```scala
161+
Ref(fooSym)
162+
```
163+
For referencing types (e.g. ones created with `Symbol.newType` or `Symbol.newClass`), use `TypeIdent` (a subtype of `TypeTree`) instead.
164+
165+
#### Flags
166+
`Flags` tell us about various attributes of `Symbols`. These can include access modifiers,
167+
whether the symbol was defined in Scala 2 or Java, whether it's `inline` or `transparent`, whether it was generated by the compiler, etc.
168+
169+
They are implemented as a bit set, with the `.is` method allowing to check if a given `Flags` is a subset, and `.|` with `.&` allowing to
170+
get a union or intersection respectively. You can see the available individual `Flags` from which to create the sets in the
171+
[api documentation](https://scala-lang.org/api/3.3_LTS/scala/quoted/Quotes$reflectModule$FlagsModule.html).
172+
173+
It's worth thinking about individual `Flags` more in terms of explicitly stated modifiers, instead of general attributes.
174+
For example, while we might say that every trait is `abstract`, a symbol of a trait will not have their `abstract` flag set
175+
(just the `trait` flag instead), simply because it does not make sense to have an `abstract trait`.
176+
177+
Different types of Symbols have different flags allowed to be set, as stated in the API docs for individual `Symbol` constructor methods.
178+
179+
### TypeReprs and TypeTrees
180+
When writing macros, we have access to `scala.quoted.Type`, which we can use to assign types in quoted code.
181+
In the context of the reflection api however, it won't be of much use. We can convert it into a more useful
182+
`TypeRepr` with `TypeRepr.of[T]` (when we have a given Type[T] in scope) which we can also convert back into a `Type`, with the simplest method being:
183+
```scala
184+
typeRepr.asType match
185+
case '[t] =>
186+
// access to a given Type[t] in scope
187+
```
188+
189+
`TypeRepr`s are a type representation used when assigning and reading types from `Symbols`. It can be constructed/read similarly to the Typed AST trees. E.g.:
190+
```Scala
191+
List[String]
192+
```
193+
is represented as:
194+
```scala
195+
AppliedType(
196+
TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),object immutable),List),
197+
List(TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class java)),object lang),String))
198+
)
199+
```
200+
Similarly to [Typed ASTs](#typed-abstract-syntax-trees), you can find the `TypeRepr` type hierarchy in
201+
[reflectModule](https://scala-lang.org/api/3.3_LTS/scala/quoted/Quotes$reflectModule.html) docs.
202+
Most of the nodes like `AppliedType` `AndType`, `MethodType`, etc. should be self explanatory,
203+
but `TypeRef` and `TermRef` might require some additional context:
204+
* `TypeRef(prefix, typeSymbol)` - corresponds to a selection of a type. E.g.: if `SomeType` is a type located in `prefix`,
205+
and `someTypeSymbol` is its `Symbol`, `TypeRef(prefix, someTypeSymbol)` will correspond to prefix.SomeType
206+
* `TermRef(prefix, termSymbol)` - corresponds to a selection on a term, which can also be useful if we are trying †o get a path dependent type.
207+
E.g.: if `someVal` is a val in `prefix`, and `someValSymbol` is its symbol, then `TermRef(prefix, someValSymbol)` will correspond
208+
to `prefix.someVal.type`. TermRef can be widened into their underlying non-TermRef type with `.widenByTermRef`.
209+
210+
Generally, if we need to insert a type directly as part of a tree (e.g. when passing it as a type parameter with a `TypeApply`),
211+
we would use a `TypeTree` (subtype of `Tree`) instead.
212+
213+
#### Extracting TypeReprs from Symbols
214+
215+
Since `TypeReprs` allow us to create and analyse `Symbols`, we might expect there to be a method to obtain the type of a `Symbol`.
216+
While there do exist `.typeRef` and `.termRef` methods, they can only generate TypeRefs or TermRefs that are usable only in
217+
the scope of it's owner. E.g. for:
218+
```scala
219+
val value: List[String] = List("")
220+
```
221+
If we were to call `.typeRef` on the symbol of value, we would get `TypeRef(This(...), valueSymbol)`, instead of `List[String]`.
222+
This is because **Symbols hold incomplete type information**.
223+
Let's look at the following:
224+
```scala
225+
class Outer[T]:
226+
val inner: List[T] = ???
227+
```
228+
The type of `inner` depends on the type parameter of `Outer` - so just having the symbol of `inner`
229+
(which has no information about its prefix, in fact the symbols of `new Outer[Int].inner` and `new Outer[String].inner` are equal) is not enough.
230+
However, we can still read the type if we have the prefixing `TypeRepr` with `prefix.memberType(symbol)` or `prefix.select(symbol)`:
231+
```scala
232+
val prefix = TypeRepr.of[Outer[String]]
233+
val innerSymbol = Symbol.classMember
234+
prefix.memberType(innerSymbol)
235+
// The above returns:
236+
//
237+
// AppliedType(
238+
// TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class collection)),object immutable),List),
239+
// List(TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class java)),object lang),String))
240+
// )
241+
```
242+
243+
### Navigating the API documentation
244+
All Quotes reflection API documentation can be found inside of the
245+
[reflectModule](https://scala-lang.org/api/3.3_LTS/scala/quoted/Quotes$reflectModule.html) trait in the scala library API docs.
246+
Due to the implementation details, methods relevant to a certain type are split between `_Module` and `_Methods` traits.
247+
For example, if we were to work on a `Select` node, the static methods like `apply` and `unapply` would be found in `SelectModule`,
248+
and methods on instances of `Select` would be found in `SelectMethods`.
71249

72250
### Positions
73251

74-
The `Position` in the context provides an `ofMacroExpansion` value. It corresponds
252+
The `Position` in the `quotes.reflect.*` provides an `ofMacroExpansion` value. It corresponds
75253
to the expansion site for macros. The macro authors can obtain various information
76254
about that expansion site. The example below shows how we can obtain position
77255
information such as the start line, the end line or even the source code at the
@@ -94,7 +272,7 @@ def macroImpl()(quotes: Quotes): Expr[Unit] =
94272
...
95273
```
96274

97-
### Tree Utilities
275+
## Tree Utilities
98276

99277
`quotes.reflect` contains three facilities for tree traversal and
100278
transformation.
@@ -118,12 +296,12 @@ def collectPatternVariables(tree: Tree)(using ctx: Context): List[Symbol] =
118296
```
119297

120298
A `TreeTraverser` extends a `TreeAccumulator[Unit]` and performs the same traversal
121-
but without returning any value.
299+
but without returning any value.
122300

123301
`TreeMap` transforms trees along the traversal, through overloading its methods it is possible to transform only trees of specific types, for example `transformStatement` only transforms `Statement`s.
124302

125303

126-
#### ValDef.let
304+
### ValDef.let
127305

128306
The object `quotes.reflect.ValDef` also offers a method `let` that allows us to bind the `rhs` (right-hand side) to a `val` and use it in `body`.
129307
Additionally, `lets` binds the given `terms` to names and allows to use them in the `body`.

0 commit comments

Comments
 (0)