Last updated: 2026-11-15
The v2 Toolchain: Compiler, Interpreter and Gates
For new readers
PatLang 2.0 is three programs, all written in PatLang and all built by the v2 compiler: the compiler itself, an interpreter, and a test-gate runner that decides what needs retesting after a change. This page is the working reference for each: the commands and options, what the compiler does in what order and how it reuses earlier work, how the interpreter runs programs and records traces for the browser debugger, and how the gates run hours of tests in parallel while leaving the machine usable.
The compiler Applied / MethodologicalKnowledge with a 5โ10 year half-life โ stable practice
v2.exe SRC.patlang OUT.exe [--workers N] [--shuffle SEED] [--no-cache]
v2.exe SRC.patlang OUT.ir --ir
v2.exe SRC.patlang OUT.wasm --wasm
v2.exe SRC.patlang --constant-time
v2.exe --evict KEEP_MB
v2.exe is self_hosting/v2/compile.patlang built by itself, at self_hosting/build/v2.exe, and runs from the repository's root. A build with errors exits 1, and a usage error 2.
| Option | Effect |
|---|---|
--workers N | Run the per-function jobs on N threads (1 by default) |
--shuffle SEED | Run the jobs in a shuffled order; the executable is byte-identical whatever the order |
--no-cache | Build every job, the runtime's included, without reading stored artefacts |
--ir | Write the program's IR as text, and nothing else (The v2 IR) |
--wasm | Write a WebAssembly module for the browser host |
--constant-time | Build nothing; report where time can depend on a secret (Constant-time arithmetic) |
--evict KEEP_MB | Remove stored jobs, least recently used first, until the store is no larger than KEEP_MB |
Stages
- Front end. Read the program and the runtime, expand
syntaxdefinitions, take each file once by its normalised path, and parse, with the lexer and parser shared with the rest of the toolchain. - Lowering (
lower.patlang). Each declaration's syntax tree becomes IR. Lowering records the program facts as it goes: the events handled, the functions reached by name, the rules. It also fits each named call to its declaration and refuses a wrong argument count. - One job per function (
jobs.patlang): the ownership stage (own.patlang), code generation (x64.patlang) and assembly (lib/x64_asm.patlang), giving the function's chunk of machine code. - Linking (
lib/x64_pe_link.patlang): the chunks, in declaration order, into a PE executable.
Each stage is a function of its inputs and touches no shared mutable state, which is what makes the parallel and the sequential build give the same bytes. The swarm compiler explains the design in full.the graph of jobs is the build's heart
The cache
A job's key hashes the function's IR, the program facts its code generation reads, the ownership summary of every function it calls, and the compiler's own key. Its artefacts are stored under self_hosting/build/v2_artefacts and reused by any later build, of any program, with the same key. So a second program reuses every runtime job, and changing one function's body rebuilds that function alone, unless its summary changes too.
- The compiler's key. When v2 builds a compiler, it writes
OUT.keybeside it: a hash of every source the compiler was built from, as they were read. The compiler reads the key beside itself and mixes it into every job key. An earlier version hashed the compiler's sources as they stood on disk, so after an edit to the code generator a compiler built before the edit reused code from the old one. A compiler with no key builds without the cache and says so. - Complete or absent. A job's record gives the length of each artefact and ends with a marker line. An artefact shorter than that, cut short by an interrupted build or by another build writing it at that moment, is built again. Two builds can share a store with no lock: a job's artefacts are the same bytes whoever writes them.
- Eviction happens only when asked, with
--evict. A build marks each job it reuses, so the least recently used go first.
Build times Ephemeral / ToolingKnowledge that evolves in months to a year โ check for updates
Measured in November 2026 on a 12-core, 20-thread machine:
| Build | Time |
|---|---|
| Warm build of a one-line program | About 12.3 s: front end 2.2, keys 1.05, jobs 2.2, link 6.7 |
| Cold build of the compiler on 8 workers | About 7.8 minutes |
| The compiler's own warm link | 22 s |
Most of the fixed cost is linking the runtime's few hundred stored functions, and it does not run in parallel. More workers help most where there are many jobs: a cold build of the compiler went from 995 seconds on one worker to 386 on eight and 326 on sixteen, while a one-line program gained little. One worker, the default, is the slowest setting in every case measured, and eight give most of the gain.
Much of v2's compile speed came from finding quadratic costs in the compiler's own PatLang: a list push that copied the list when its value read the list, a linker that gave every name index 4,096 buckets, an assembler that found each byte of a value by repeated subtraction, and an artefact reader that turned each byte into a one-character string. Each fix kept the executables byte-identical, which the fixpoint check confirmed.
The interpreter Applied / MethodologicalKnowledge with a 5โ10 year half-life โ stable practice
v2_interp.exe SRC.patlang [args...]
The interpreter is self_hosting/v2/interp.patlang, a command over the library self_hosting/v2/interpreter.patlang, built by v2 as self_hosting/build/v2_interp.exe. It finds the runtime's source from its own executable's path, so it must stay in self_hosting/build/, and a program gives the same result whichever directory it is started in.
It lowers the program with the compiler's own front end, against the runtime's declarations alone, and calls the runtime's compiled functions directly, so only the program's own code is interpreted. Each function is translated once into a list of integers read word by word, with each operand held as its byte offset in the frame and each jump as its target's byte offset, and its constants in a second list. Frames come from one block the interpreter allocates, not a list per call, and two ints add directly when no handler is registered. Those changes took bench_fib from 104.9 seconds to 8.9; startup takes about 1.7 seconds.frame is a block of memory for local variables
Traces and the debugger
Asked to, the lowering compiles a program to report each step: it calls __trace_step before each statement with its line, and __trace_enter and __trace_leave around each function's body, with the function and its locals. A traced run gives those names closures that record into a trace. The IDE's debugger steps through such a trace after the program has run, showing the line, the function and its locals at each step.
In the browser
The playground, the REPL, the IDE and the dynamic-syntax demo all run lib/interp_run.patlang's world backend, built for the browser by v2. A page gives it the runtime's files as a read-only bundle, and it compiles the user's text with the front end and runs it in a fresh world on the interpreter. Every demo card in the portfolio is built by v2, natively for its transcript and as WebAssembly for its page.
The gates Applied / MethodologicalKnowledge with a 5โ10 year half-life โ stable practice
self_hosting/v2/tools/gates.sh [--all] [--plan] [--cap N] [--only PREFIX]
The gates are every check that must pass before a change is pushed: the compiler and its fixpoint, the bootstrap reader, the WebAssembly writers, the interpreter, and each suite's feature files on each engine. Run one after another, they took 4 hours 52 minutes. gates.sh builds the first v2.exe from the IR when there is none, uses it to build the gates tool, self_hosting/v2/tools/gates.patlang, and runs the tool with its own arguments.
| Option | Effect |
|---|---|
| (none) | Run the units that are stale, in parallel |
--all | Run every unit, fresh or not, for before a push and to check the record against reality |
--plan | Say what would run, and run nothing |
--cap N | Give the units N cores whatever else the machine is doing |
--only PREFIX | Keep as goals the units whose names start with PREFIX; what they need still runs when it must |
Units, keys and the plan
A unit is one build or one feature file of one suite on one engine. Each declares the units it needs, the files and directories its result depends on (a runner's source with every file it includes, and the fixtures and directories its steps read), the cores it keeps busy, and a lane: units in one lane never run together, as the language suite's two engines share ports and files. A unit's key hashes its inputs' contents and the keys of the units it needs. A unit that passed with its current key is fresh and is not run again; a unit with no declared run-time inputs is always stale, so a missing declaration costs time and never gives a wrong verdict.
The tool uses PatLang's own logic engine for this. The units, their needs and inputs are facts, and staleness is a rule. The stale units are the planner's goals, its actions are the units, and each action needs what the units it depends on produce, so a build that no stale unit needs, and whose product is present, does not run. Running the gates this way found the old planner's habit of answering "no plan" for problems that had one, and led to the A* planner.stale means: run me again
Sharing the machine
- Cores. Units get at most the logical cores less two, by weight. Before starting one, the tool reads
cpu_times; cores busy beyond what its own units account for belong to someone else, and the budget shrinks by them, never below one unit at a time. - Memory. A unit starts only when
memory_statusshows room for its peak from last time (or 2 GB) plus the peaks of the units running and a 4 GB reserve for the user. Peaks are measured withexec_measured, which runs the command in a Windows job object, so a suite's compiles and child programs count. - Isolation. Each unit writes only under
self_hosting/build/gates/out/UNIT, given to the runner asPATLANG_SPEC_OUT, and fixtures that listen on the network take port 0, so two units never collide. - Status. Progress goes to
self_hosting/build/gates/status, and astatussignal on the port that file names gives the same. Aquitsignal starts no more units and lets the running ones finish; each result is recorded as it comes, so the next run takes up the rest.
Results Ephemeral / ToolingKnowledge that evolves in months to a year โ check for updates
| Run | Time |
|---|---|
| Every gate, one after another | 4 h 52 min |
| The other 146 units in parallel, with Part 1 already fresh | 2 h 36 min |
| Part 1 on both engines, in parallel, from nothing | About 28 min, against 83 sequentially |
| A second run with nothing changed | 4 s, running nothing |
After a blank line added to text.feature | Exactly that feature's two units |
Every suite gave its sequential count in the parallel run: Part 2 136, the language suite 297 on each engine, match 26, the block model 210, engines agree 62, the interpreter-run selftest 54 and the isolate selftest 65, with the bootstrap and WebAssembly checks identical. Two feature files now bound a full run, each longer than its whole suite took alone, since everything else competes with them for cores; they start first so the rest fits inside them.
Rust is out of the loop Ephemeral / ToolingKnowledge that evolves in months to a year โ check for updates
At the pre-v2 tag, three things depended on Rust: pat --ir-run ran the compiler, the suite runners and everyday development; WebAssembly went through generated Rust and rustc; and a clean checkout started from a Rust build. All three are gone. Since 14 November 2026 no step of any suite starts pat; the site builder and the portfolio builder run compiled by v2; the browser gets its modules from v2's own writer; and a clean checkout starts from the IR and a Python reader. The Patbuild demo page, which built its targets with rustc, was retired with it.v2 compiles itself, bootstrapping the toolchain
The old compiler's runtime and fixtures still keep their own global state, and leave with it when it is retired. When the 418 programs that existed at the tag were surveyed a second time, on 6 November, 385 compiled on v2. Of the 33 that did not, 16 stopped on the name unit, which has been fixed since; the rest were include paths that no longer exist, three parse errors, examples written to test the Rust runtime itself, and one other.
See also
The swarm compiler covers the job graph and the cache in depth. Six self-hosted dev tools and the BDD framework describe the tools the suites are written with. What changed since v1 summarises the release.
Graduate Attributes
Modelled on this page
- Subject Specialist: Technical details of compiler stages, caching keys and interpreter frames are presented with precise implementation specifics.
- Researcher: Performance improvements are substantiated with specific benchmark timings before and after optimisations.
- Critical Thinker: Build time data is analysed to identify diminishing returns on parallelism and justify default worker settings.
University of Reading Graduate Attributes framework.