Skip to content

Commit e92be9e

Browse files
aapoalasaapoalas
authored
chore(nova): Update README.md, add CONTRIBUTING.md and ARCHITECTURE.md (#176)
1 parent 79b29f3 commit e92be9e

3 files changed

Lines changed: 209 additions & 41 deletions

File tree

‎ARCHITECTURE.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Architecture
2+
3+
The architecture of Nova engine is built around data-oriented design. This means
4+
that most "data" like things are found in the heap in a vector with like-minded
5+
individuals.
6+
7+
## Concurrency
8+
9+
Currently the heap is not thread-safe at all but the plan is to make it
10+
thread-safe enough for concurrent marking to become possible in the future. To
11+
this end, here's how I (@aapoalas) envision the engine to look like in the
12+
future:
13+
14+
```rs
15+
struct Agent<'agent, 'generation>(Box<AgentInner<'agent>>, AgentGuard<'generation>);
16+
```
17+
18+
The `Agent` struct here is just a RAII wrapper around the inner Heap-allocated
19+
Agent data. The first important thing is the `'agent` lifetime: This is a brand.
20+
It is valid for as long as the Nova engine instance lives, and its only purpose
21+
is to make sure that (type-wise) uses cannot mistakenly or otherwise mix and
22+
match Values from different engine instances.
23+
24+
The second lifetime, `'generation`, is the garbage collection generation. Here
25+
what I want to achieve is a separation between "gc" and "nogc" scopes. But
26+
before we dive into that, here's what I imagine a `Value` looking like:
27+
28+
```rs
29+
enum Value<'generation> {
30+
Undefined,
31+
String(StringIndex<'generation>>),
32+
SmallString(SmallString),
33+
// ...
34+
}
35+
```
36+
37+
The `Value` enum carries the `'generation` lifetime: As long as we can guarantee
38+
that no garbage collection happens, we can safely keep `Value<'gen>` on the
39+
stack or even temporarily on the heap.
40+
41+
If we call a method that may trigger GC, then all `Value<'gen>` items are
42+
invalidated. If we want to keep values alive through eg. JavaScript function
43+
calls, we must use:
44+
45+
```rs
46+
struct ShadowStackValue<'agent>(u32, PhantomPinned);
47+
```
48+
49+
This just moves the `Value` onto an Agent-controlled "shadow stack" that the
50+
`u32` points into. Due to the `PhantomPinned` the shadow stack is mostly just
51+
push-pop as any stack should be, and thus relatively quick. But it is also on
52+
the heap and thus garbage collection can update any references on the shadow
53+
stack.
54+
55+
Note that this is essentially equivalent to:
56+
57+
```rs
58+
struct GlobalValue<'agent>(u32);
59+
```
60+
61+
but "global values" are not push-pop, likely will have generational indexes,
62+
possibly will have reference counting and so on and so forth.

‎CONTRIBUTING.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Contributing
2+
3+
First be forewarned: Contributions are wanted and dearly desired, but @aapoalas
4+
is a pain to deal with. He'll nitpick your PR to the bone if he happens to be in
5+
the mood to do so. He's not doing it to be evil or anything, he's just an idiot.
6+
Please forgive him, if you can.
7+
8+
The second warning is: There is currently no license in place for Nova. The
9+
license will be some sort of open source license, but whether it'll be some
10+
copyleft license or just MIT is still undecided. At the end of the day,
11+
@aapoalas (again with that guy) has some aspirations of making this project be
12+
bigger than just a hobby-engine. If that day comes, it'd be nice to get more
13+
than a warm handshake out of it. Hence the ongoing lack of license.
14+
15+
## List of active development ideas
16+
17+
Here are some good ideas on what you can contribute to.
18+
19+
### Technical points
20+
21+
The heap will need to be concurrently marked at some point. Additionally, we'll
22+
want to split some heap data structures into two or more parts; only the
23+
commonly used parts should be loaded into L1 cache during common engine
24+
operations.
25+
26+
For this purpose we'll need our own `Vec`, `Vec2`, `Vec3` and possibly other
27+
vector types. The first order of business is to get the length and capacity to
28+
be stored as a `u32`. The second will be enabling the splitting of heap data
29+
structures; this sbould work in a way similar to `ParallelVec` so that the size
30+
of `Vec2` and `Vec3` stays equal to `Vec`.
31+
32+
Then finally, at some point we'll also want to make the whole heap thread-safe.
33+
Heap vectors (`Vec`, `Vec2`, ...) will become RCU-based, so when they expand (on
34+
push) they will return a `None` or `Some(droppable_vec)` which can either be
35+
dropped immediately (if concurrent heap marking is not currently ongoing) or
36+
pushed into a "graveyard" `UnsafeCell<Vec<(*mut (), fn(*mut ()))>>` that gets
37+
dropped at the end of a mark-and-sweep iteration.
38+
39+
These and other technical work items can be found from the GitHub issues with
40+
the
41+
[`technical` label](https://github.com/trynova/nova/issues?q=is%3Aopen+is%3Aissue+label%3Atechnical+).
42+
43+
### Internal methods of exotic objects
44+
45+
ECMAScript spec has a ton of exotic objects. Most of these just have some extra
46+
internal slots while others change how they interact with user actions like
47+
get-by-identifier or get-by-value etc.
48+
49+
You can easily find exotic objects' internal methods by searching for
50+
`"fn internal_get_prototype_of("` in the code base. Many of these matches will
51+
be in files that contain a lot of `todo!()` points. As an example,
52+
[proxy.rs](./nova_vm/src/ecmascript/builtins/proxy.rs) is currently entirely
53+
unimplemented. The internal methods of Proxies can be found
54+
[here](https://tc39.es/ecma262/#sec-proxy-object-internal-methods-and-internal-slots):
55+
These abstract internal methods would need to be translated into Nova Rust code
56+
in the `proxy.rs` file.
57+
58+
[This PR](https://github.com/trynova/nova/pull/174) can perhaps also serve as a
59+
good guide into how internal methods are implemented: Especially check the first
60+
and third commits. One important thing for internal method implementations is
61+
that whenever a special implementation exists in the spec, our internal method
62+
should link to it. Another thing is that if you cannot figure out what you
63+
should be calling in the method or the method you should be calling doesn't
64+
exist yet and you think implementing it would be too much work, it is perfectly
65+
fine to simply add a `todo!()` call to punt on the issue.
66+
67+
### Builtin functions
68+
69+
Even more than internal methods, the ECMAScript spec defines builtin functions.
70+
The Nova engine already includes bindings for nearly all of them (only some
71+
Annex B functions should be missing) but the bindings are mostly just `todo!()`
72+
calls.
73+
74+
Implementing missing builtin functions, or at least the easy and commonly used
75+
parts of them, is a massive and important effort. You can find a mostly
76+
exhaustive list of these (by constructor or prototype, or combined)
77+
[in the GitHub issue tracker](https://github.com/trynova/nova/issues?q=is%3Aopen+is%3Aissue+label%3A%22builtin+function%22).
78+
79+
### Other things
80+
81+
This list serves as a "this is where you were" for returning developers as well
82+
as a potential easy jumping-into point for newcompers.
83+
84+
- Write implementations of more abstract operations
85+
- See `nova_vm/src/ecmascript/abstract_operations`
86+
- Specifically eg. `operations_on_objects.rs` is missing multiple operations,
87+
even stubs.
88+
89+
Some more long-term prospects and/or wild ideas:
90+
91+
- Reintroduce lifetimes to Heap if possible
92+
- `Value<'gen>` lifetime that is "controlled" by a Heap generation number:
93+
Heap Values are valid while we can guarantee that the Heap generation number
94+
isn't mutably borrowed. This is basically completely equal to a scope based
95+
`Local<'a, Value>` lifetime but the intended benefit is that the
96+
`Value<'gen>` lifetimes can also be used during Heap compaction: When Heap
97+
GC and compaction occurs it can be written as a transformation from
98+
`Heap<'old>` to `Heap<'new>` and the borrow checker would then help to make
99+
sure that any and all `T<'new>` structs within the heap are properly
100+
transformed to `T<'new>`.
101+
- Add a `Reference` variant to `Value` (or create a `ValueOrReference` enum that
102+
is the true root enum)
103+
- ReferenceRecords would (maybe?) move to Heap directly. This might make some
104+
syntax-directed operations simpler to implement.
105+
- Add `DISCRIMINANT + 0x80` variants that work as thrown values of type
106+
`DISCRIMINANT`
107+
- As a result, eg. a thrown String would be just a String with the top bit set
108+
true. This would stop Result usage which is a darn shame (can be fixed with
109+
Nightly features). But it would mean that returning a (sort of)
110+
`Result<Value>` would fit in a register.
111+
- Consider a register based VM instead of going stack based

‎README.md‎

Lines changed: 36 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -14,48 +14,43 @@ more details.
1414

1515
The core of our team is on our [Discord server](https://discord.gg/RTrgJzXKUM).
1616

17-
## Heap structure
17+
## [Architecture](./ARCHITECTURE.md)
1818

19-
If you find yourself interested in what on earth is going on in the Heap, take a
19+
The architecture of the engine follows the ECMAScript specification in spirit,
20+
but uses data-oriented design for the actual implementation. Records that are
21+
present in the specification are likely present in the Nova engine as well and
22+
they're likely found in an "equivalent" file / folder path as the specification
23+
defines them in.
24+
25+
Where the engine differs from the specification is that most ECMAScript types
26+
and specification Record types are defined "twice": They have one "heap data"
27+
definition, and another "index" definition. The heap data definition generally
28+
corresponds to the specification's definition, in some degree at least. The
29+
index definition is either a wrapper around `u32` or a `NonZeroU32`. Most spec
30+
defined methods are defined on the index definitions (this avoids issues with
31+
borrowing).
32+
33+
The only case when direct "Record type A contains Record type B" ownership is
34+
used is when there can be only one referrer to the Record type B.
35+
36+
### Heap structure - Data-oriented design
37+
38+
Reading the above, you might be wondering why the double-definitions and all
39+
that. The ultimate reason is two-fold:
40+
41+
1. It is an interesting design.
42+
2. It helps the computer make frequently used things fast while allowing the
43+
infrequently used things to become slow.
44+
45+
Data-oriented design is all the rage on the Internet because of its
46+
cache-friendliness. This engine is one more attempt at seeing what sort of
47+
real-world benefits one might gain with this sort of architecture.
48+
49+
If you find yourself interested in where the idea spawns from and why, take a
2050
look at [the Heap README.md](./nova_vm/src/heap/README.md). It gives a more
2151
thorough walkthrough of the Heap structure and what the idea there is.
2252

23-
## List of active development ideas
24-
25-
This list serves as a "this is where you were" for returning developers as well
26-
as a potential easy jumping-into point for newcompers.
27-
28-
- Write implementations of more abstract operations
29-
- See `nova_vm/src/ecmascript/abstract_operations`
30-
- Specifically eg. `operations_on_objects.rs` is missing multiple operations,
31-
even stubs.
32-
- Write implementations of class abstract operations
33-
- String, Number, ...
34-
- Start `nova_vm/src/syntax` folder for
35-
[8 Syntax-Directed Operations](https://tc39.es/ecma262/#sec-syntax-directed-operations)
36-
- This will serve as the bridge between oxc AST, Bytecode, and Bytecode
37-
interpretation
38-
39-
Some more long-term prospects and/or wild ideas:
40-
41-
- Reintroduce lifetimes to Heap if possible
42-
- `Value<'gen>` lifetime that is "controlled" by a Heap generation number:
43-
Heap Values are valid while we can guarantee that the Heap generation number
44-
isn't mutably borrowed. This is basically completely equal to a scope based
45-
`Local<'a, Value>` lifetime but the intended benefit is that the
46-
`Value<'gen>` lifetimes can also be used during Heap compaction: When Heap
47-
GC and compaction occurs it can be written as a transformation from
48-
`Heap<'old>` to `Heap<'new>` and the borrow checker would then help to make
49-
sure that any and all `T<'new>` structs within the heap are properly
50-
transformed to `T<'new>`.
51-
- Add a `Reference` variant to `Value` (or create a `ValueOrReference` enum that
52-
is the true root enum)
53-
- ReferenceRecords would (maybe?) move to Heap directly. This might make some
54-
syntax-directed operations simpler to implement.
55-
- Add `DISCRIMINANT + 0x80` variants that work as thrown values of type
56-
`DISCRIMINANT`
57-
- As a result, eg. a thrown String would be just a String with the top bit set
58-
true. This would stop Result usage which is a darn shame (can be fixed with
59-
Nightly features). But it would mean that returning a (sort of)
60-
`Result<Value>` would fit in a register.
61-
- Consider a register based VM instead of going stack based
53+
## [Contributing](./CONTRIBUTING.md)
54+
55+
So you wish to contribute, eh? You're very welcome to do so! Please take a look
56+
at [the CONTRIBUTING.md](./CONTRIBUTING.md).

0 commit comments

Comments
 (0)