The Journey of Building PatLang, Continued Still Further: Taking the Native Backend Seriously

For new readers

This is one instalment in an ongoing, chronological diary of building PatLang, written up in numbered "Acts" as the project actually happened, warts included. You don't need to have read the earlier instalments to follow this one, but it helps to know that PatLang already compiles two ways: interpreted directly, or lowered and handed to rustc to produce a real executable. Neither of those is a genuine PatLang-to-native compiler — rustc is still doing the actual machine-code generation underneath. A separate, mostly-finished experiment does exist that skips rustc entirely and emits real x64 machine code directly — and, tellingly, it's written in PatLang itself, not Rust, matching the project's own stated long-term direction. This instalment is the story of taking that experiment out of storage and actually trying to trust it.

A direct continuation of the previous instalment (Acts XXIX-XXXV: teaching the language's own GOAP planner and induction engine to find working programs by search, three separate vacuous results caught before being written up as wins, and a quieter arc of ordinary site maintenance that turned up two genuine runtime bugs along the way) — split into its own page for the same reason each earlier page split off from the one before it. This arc starts with a scoping question about an open bug report, and ends up somewhere much more concrete: a real, if partial, working native backend, several genuinely dangerous silent-wrong-answer bugs found and fixed in it, one moment where the project's own stated direction had to correct an answer already given, and — several acts later, after closing the backend's entire remaining primitive gap list and surviving a fourteen-gigabyte assembler process along the way — the milestone the whole effort had been aiming at: a compiler compiled by this backend, successfully compiling itself. That milestone turns out, three acts later, to have been real but incomplete — a genuine class/object system and a parallel per-function compile cache built on top of it, issue #31's actual two-part root cause finally found and fixed, a second-generation self-hosting crash caught only by pushing the fixpoint test further than strictly required, a string-representation proposal weighed fairly and declined, and the backend's entire remaining primitive gap list closed bar one deliberately-excluded item — verified, this time, across a genuine four-generation self-compiling chain rather than just one.

Act XXXVI: "we do NOT want Rust backends"

The starting question was an open GitHub issue: PatLang's compiled programs still run through an interpreter loop under the hood, even when handed to rustc — not genuine native codegen. A quick reality check made the scope bigger before it got smaller: "we may not need #1 for speed on a native machine but x64 is not going to be relevant when compiling to wasm, for instance" — correctly pointing out that a native x64 backend and a WASM backend are fundamentally different targets, so finishing one could never resolve the other. Asked how to scope this, the answer that came back added a real architectural requirement rather than just narrowing the ticket: "option 1 but making sure the x64 generator and (potential) wasm (or indeed, other targets) generator(s) are modular steps which act as drop in replacement options for each other in the pipeline."

Investigating what already existed turned up something more interesting than the issue's own text implied. The x64 backend wasn't an abandoned Rust module sitting half-finished next to the interpreter — it was a wholly separate, self-hosted PatLang implementation: its own lexer, parser, lowerer, and NASM-emitting codegen, all written in PatLang, invoked by running a driver script through the ordinary interpreter, never touching the compiler's real Rust Program/Function/Instr types at all. Reporting this back, the natural-sounding next step was floated: port that ~2,900 lines of NASM-emission logic into real Rust against the shared IR, so it could plug into the same dispatch as the interpreter and the rustc-backed path. That recommendation went out in writing, on the public issue tracker, as "the architecturally clean answer."

It was wrong, and the correction arrived directly: "We do NOT want Rust backends though - everything should move towards patlang implementations." The project's own standing philosophy says exactly this already — PatLang is deliberately pushed to self-host, with even the very last dependency on rustc (as a native-codegen backend) named as something meant to go away eventually, not something to lean into further. Porting the x64 backend into Rust wouldn't have been a step toward that; it would have been a step backward, dressed up as tidiness. The recommendation had to be walked back in writing, on the same public thread, rather than quietly dropped: any future backend-selection mechanism should be a thin dispatcher deciding which self-hosted PatLang backend to run through the interpreter — not a place to write or grow backend logic in Rust. Lesson: "architecturally clean" is not a neutral engineering judgement — it can smuggle in a language-choice preference that directly contradicts a project's own stated direction, and the only fix, once posted, is a visible correction, not a quiet edit.

Act XXXVII: a modulo operator, a sign flip, and a bug hiding behind another bug

With the direction corrected, the actual work became: does this half-finished native backend hold up under real use? The starting probe was narrow — "If the grammar supports % for numerics, the code generator should also support it" — and immediately found that float % wasn't implemented at all: it printed a diagnostic, then emitted broken assembly anyway, cascading into a wall of unrelated-looking NASM syntax errors rather than failing cleanly. Fixed with a real floating-point remainder sequence. The follow-up question went straight to the heart of the backend's own honesty: "check the numeric tower handles % correctly too" — and found something worse than a missing feature. A function that happened to also touch BigInt values silently fell back to plain integer arithmetic on a BigInt's raw heap-pointer bits whenever % was used, with no error at all — correct by accident for small numbers, silently wrong for a real BigInt. Since genuine arbitrary-precision modulo didn't exist to fall back to, the fix was a correct fast path for the common case plus a loud, distinctive runtime failure for the case that genuinely couldn't be handled yet — a wrong answer turned into a visible one.

Fixing the crash trail from the float bug surfaced something much more serious sitting right next to it: unary negation of a float — plain -x — was using integer two's-complement negation on the value's raw bit pattern. That is not a sign flip in general; the carry from that operation can propagate straight into a floating-point number's exponent bits. Concretely, -1.5's bit pattern, negated as if it were a plain integer, silently became the bit pattern for -3.0 — twice too large, in the wrong direction of nobody's expectation, with no crash and no diagnostic. It is exactly the kind of bug that survives for a long time precisely because the output still looks like a plausible number. Fixed with a proper sign-bit flip instead of an integer negation.

Verifying the fix meant deliberately mixing floats and closures in one small test program, and that test produced its own, unrelated garbage number — apply_twice(double, 5), which should print 20, printed a sixteen-digit number instead. Investigated on its own terms first: was this a closure bug? A minimal repro with no floats anywhere ran correctly. So what changed when a float was added to the same function? The backend classifies an entire function as either all-float or all-int, never mixed — and the plain integer literal 5, sharing a function with an unrelated float literal, was being encoded as the bit pattern for 5.0. That value then crossed into a genuinely plain-integer function, which multiplied it as an integer — 5.0's bit pattern, doubled twice, wraps around a 64-bit register to land on precisely the garbage number that had been observed. Traced by hand, confirmed to the last digit. A loud compile-time warning went in first as an honest stopgap, then a real, narrower fix: an integer literal passed directly as an argument to a known, statically-resolvable plain-integer function now keeps its integer encoding regardless of what else that particular function happens to contain — filed and closed as its own tracked issue, since the user's own framing of it — "fix the nested-closure bug, with its own issue as that is a nasty" — was exactly right about how easily this specific shape of bug could hide.

A genuinely separate closure bug turned up in the same stretch, this time nothing to do with x64 at all: a closure that captured an outer variable and then shadowed it on its very first use — let n = n + 1, where the right-hand n needs the captured outer value — silently read zero instead, in the plain interpreter, on any backend. The variable-capture analysis treated every name a closure ever declares, anywhere in its body, as fully local, with no notion of "before this point" versus "after" — so the one read that genuinely needed the outer value got excluded along with every other, unrelated redeclaration. Fixed by specifically recognising a name's own self-referential first declaration and forcing capture for exactly that case. Lesson: fixing one bug can make its own log output honest enough to reveal a second, completely unrelated bug sitting right behind it — and "the caller function also happens to contain a float literal" is not a detail to shrug off, if the backend's own typing decisions are made per whole function rather than per value.

Act XXXVIII: the print bug that wasn't a print bug

Confirming the fixes hadn't broken anything else meant re-running the very first small test program from this whole arc — recursive Fibonacci numbers, a string built with plain +, and a small list — and it very nearly passed. The Fibonacci numbers were right. The list was right. The string printed <value> instead of hello world. The natural first guess, given everything else touched so far, was a print-formatting regression somewhere in the same code being edited. It wasn't. The instinct about where the real weight was turned out to be exactly right too: "Should make an issue for the print problem to get it resolved. And I suspect that is the highest value fix to address."

Tracing it properly found something much bigger than a display quirk: ordinary string concatenation via + had never actually been implemented in this backend at all, in either of its two internal code paths. A function that concatenates strings but doesn't otherwise touch the numeric tower was never being classified as needing the tag-aware dispatch strings require, so + fell through to plain integer addition on the two strings' tagged pointers — and two string-tagged addresses added together happen to produce a bit pattern that collides with a completely different value's own tag, an internal coincidence with no relation to the string's actual content. The print routine's own dispatcher, seeing a tag it didn't recognise, correctly fell back to its placeholder — which is what made a fundamental missing feature look, from the outside, like a cosmetic print bug. Even the code path meant to handle exactly this case, once actually reached, turned out to be an unfinished stub that had never done anything but return zero. The real fix touched both: functions containing string literals are now correctly routed to the tag-aware path, and that path now genuinely concatenates two strings, rather than silently discarding one of them.

Rerunning the same original test program afterward matched the ordinary interpreter's output exactly, line for line, for the first time. A broader sweep for the same shape of mistake, prompted only by "test more broadly before calling it done," found the sibling bug immediately: string equality and ordering comparisons were also comparing raw memory addresses rather than actual text — two separately-created strings holding the identical word compared as unequal, and "less than" compared meaningless heap addresses instead of any real alphabetical order. The backend's own existing comment even called this "deliberate," which was simply incorrect against how the real language already behaved elsewhere. Fixed the same way: real content comparison for equality, and a genuine character-by-character ordering comparison for everything else, both verified directly against the plain interpreter's own answers rather than assumed. Lesson: when a "cosmetic" bug is reported alongside strong misgivings that it might actually be the important one, that instinct is worth following all the way to the bottom before settling for the smaller explanation — a print routine correctly failing to recognise garbage data is not the same claim as a print routine being broken.

Act XXXIX: merging a branch nobody had touched in a week, and a benchmark suite that found two more bugs before it even finished being built

With the standalone backend now noticeably more trustworthy, the next question was structural: an entire feature branch containing real, working x64-backend improvements — separate-compile chunk-linking, real file I/O, a register-allocator correctness fix — had been sitting unmerged for over a week. "Also need to look at getting the x64 into merged into main which may require bringing the x64 branch uptodate with main without breaking anything first?" Investigating first, rather than attempting the merge blind, turned up a pleasant surprise: nearly everything on that branch was already shared history with the main line. Exactly one commit stood apart — and it fixed almost the identical bug this arc had just fixed independently, but more thoroughly: it covered a code path this session's own fix had missed entirely, and it closed a genuine crash risk (a bare internal tag value that could be mistaken for a real piece of text under just the right conditions) that this session's own fix hadn't even considered. Rather than picking one version over the other, the better parts of both were combined — the missed code path adopted from the older branch, the crash-safety guard ported across to the code this session had written — verified against everything both branches had ever been tested with before being folded permanently into the main line and the old branch retired.

The final piece was infrastructure rather than a bug fix: building a benchmark suite that runs the same set of representative programs through all three ways PatLang can execute code — directly interpreted, compiled via the existing Rust-backed pipeline, and compiled via this session's own hardened native backend — timing each and checking that they all agree on the actual answer, with the explicit request that the newer, less-proven backend should never be allowed to stop the whole suite: "all three with safe (ie continue on fault) treatment of x64 while logging the fault for fixing." Built and run for the first time, it did exactly what a good test suite is supposed to do to code that has never been examined this closely before: found something wrong immediately. A three-way if/elif/else chain was silently executing its final else branch roughly twice as often as it should — traced to a genuine, previously invisible parser bug: the word "elif" is recognised internally as its own distinct token, but the code responsible for knowing when a branch's body has ended was only ever checking for a different, more generic kind of token, so it quietly skipped straight past the word "elif" instead of stopping there. Every plain if/else in the entire language had been working perfectly the whole time; it took a three-way branch, exercised by a genuinely new kind of test, to expose that the middle case had never actually been handled correctly. Fixed with a small, targeted addition rather than a rewrite, verified against a minimal nine-iteration repro, and confirmed to leave every existing test passing.

A second, quieter false alarm turned up moments later: two supposedly-mismatched benchmark results that looked, printed side by side, completely identical. A byte-level comparison found the actual difference immediately — one single invisible trailing blank line the interpreter's own output happens to carry that a standalone compiled program's output doesn't. The existing text-trimming helper already used throughout the codebase didn't catch it, for a reason worth keeping on record rather than quietly working around forever: it was deliberately written to treat only spaces, tabs, and carriage returns as whitespace, not newlines — whether that was always intentional or simply an oversight from whenever it was first written is now an open, explicitly flagged question, rather than something silently patched over in one more place and left for someone else to wonder about later. Lesson: a brand-new kind of test doesn't just catch brand-new kinds of bugs — a benchmark suite built for entirely different reasons (timing figures, cross-backend confidence) found two genuine, previously-invisible bugs in areas that had been considered settled for a very long time, purely by exercising a combination of language features nothing had happened to combine quite that way before.

Act XL: the roadmap's own remaining gap list, closed one primitive at a time

With the backend merged and trustworthy, the next milestone was concrete rather than aspirational: get the project's own self-hosted, meta-circular interpreter — the PatLang program that interprets PatLang, itself written in PatLang — to actually compile through the native x64 backend, not just the small hand-picked test programs used so far. The first attempt didn't get far before failing on a long list of "unsupported primitive" errors: process lifecycle (spawn/is_alive/wait/kill/sleep_ms), real TCP networking (tcp_listen/tcp_connect/tcp_accept/tcp_read/tcp_write and friends), a math library, bitfield operations, a virtual filesystem layer, and a handful of smaller reflection utilities — the entire remaining surface of host functions the interpreter's own source code happened to reach for, none of which the native backend had ever needed to support before. Asked directly whether the networking and process primitives should be genuine implementations or acceptable stubs, given that a compiler compiling itself almost certainly never opens a real socket: "Real implementation now." Not the cheaper answer, but the honest one — a stub that happens to never get called is indistinguishable, by testing, from a stub that's silently wrong the one time it does.

Each primitive went in through the same discipline used throughout this whole arc: real semantics first (reusing the exact Win32 CreateProcess/PROCESS_INFORMATION pattern the backend's own exec_capture support already proved out, for the process-lifecycle half; genuine Winsock2 calls — WSAStartup, socket, bind, listen, accept, connect, send, recv — for the networking half, with no existing self-hosted groundwork to build from at all), verified against the plain interpreter's own real behaviour on the identical scenario before being trusted, not just checked for "didn't crash." A real loopback client and server, bound, connected, and exchanging an actual message both ways in the same test run, is a meaningfully stronger claim than a socket call that merely returns without an error code. Along the way, this pass also turned up and fixed several more of the same dangerous silent-wrong-answer class of bug this arc had already found repeatedly: a negative-number arithmetic path that silently stripped the sign bit off a result PatLang-wide, a string comparison that only checked one of its two operands for being a real string before treating raw memory addresses as equal, and — a near-identical sibling bug to Act XXXVIII's string-equality fix — the ordering comparisons (<, <=, >, >=) had never been given any string-awareness at all, comparing heap addresses under a plain < on two strings the same way the equality bug once had. Six commits landed this stage's real implementations; the interpreter's list of missing primitives shrank to zero.

Act XLI: a 14-gigabyte assembler process, and the mystery that followed it

The actual milestone this whole roadmap was aiming at was bigger than the interpreter alone: getting the entire self-hosted compiler — its own lexer, parser, lowerer, and code generator, several hundred functions bundled together — to compile through the native backend for the first time. The first attempt didn't get far before something alarming showed up directly, caught not by a test but by a human watching the machine: "There are two nasms instances running - both with very large memory footprints 117580 and 51828 are their PIDs" — two assembler processes, each consuming well over ten gigabytes, for a compile that should have taken seconds. The root cause, once found, made sense in hindsight: this backend emits one NASM label per IR instruction, and several hundred functions bundled into a single assembly file pushed the label count high enough to hit a real scaling edge case in NASM's own symbol table. Asked to explain why splitting code generation into chunks — rather than the whole compiler at once — would even be possible given how much shared state the two paths seemed to need, the answer clarified that only the final NASM/linking stage needed the whole bundle in one place, not the lowering or codegen logic itself, which prompted the direct approval to proceed: "OK proceed with your suggestion." The fix split the giant function list into fixed-size chunks, assembling and linking each separately before combining the resulting object files — bringing memory use back down to normal and, as a side effect, fixing a second bug already lurking in the multi-file link step's own use of a shell command line for a very long list of object-file paths, replaced with a linker response file instead.

That fix immediately traded one failure for a stranger one: NASM now reported a label whose computed address "changed during code generation" — the classic signature of an assembler optimizer pass failing to converge, and exactly the kind of bug that invites hours of investigation into the wrong layer. Several genuinely reasonable-sounding leads were tried and each, in turn, ruled out directly rather than assumed: splitting the backend's own sprawling, deeply-nested string-comparison dispatch chain into smaller functions bucketed by first letter — a restructuring worth doing on its own merits, prompted by "Could you not split that (already noticed as nasty) chain of if else's up... There has to be a better way (or, hashing them...)?" — didn't fix it. Forcing every conditional jump in the output to its explicit long-form encoding, on the direct suggestion "OK if we forced long jumps, would that fix it for now?", didn't fix it either, confirmed by disassembling the actual output and finding the long form already present. Every one of NASM's own optimization levels, tried in turn per "have you tried all the optimisation options nasm has... Alternatively, size padding of dodgy looking opcodes, perhaps?", still failed the same way — including the least aggressive setting, which should never trigger a convergence bug at all if the cause really were an optimizer subtlety.

The actual crack in the case came from a single, sharply-aimed observation aimed at a completely different part of the error output: "I notice is it happening just after it says interpret_ir is not defined... might that be a problem?" It was the whole problem. The new chunk-splitting function's own trailing-chunk logic — flush whatever functions are left over once the loop finishes — used a length check that silently misbehaved specifically when running through the compiled backend rather than the interpreter (a bare comparison against a list length, missing a numeric-coercion call this codebase's own convention otherwise applies everywhere), and had been quietly dropping its own final chunk from the compile entirely. That final chunk happened to contain interpret_ir itself — so a call site elsewhere in the program was referencing a symbol that genuinely didn't exist anywhere in the assembled output, and NASM's own diagnostics for that situation are a misleading multi-pass "label changed" cascade rather than a clean "undefined symbol" error pointing at the one real line responsible. Confirmed by toggling the single missing coercion call on and off: broken gives three chunks and dozens of assembler errors, fixed gives four chunks and zero. Four hours had gone into chasing an assembler optimizer bug that never existed, and the actual root cause was a one-token omission in code written the same session. The feedback that followed was blunt and entirely earned: "Looks like this would have been spotted much more quickly (4 hours and 15% of my weekly token allowance) if you'd put proper BDD in place - when compiling a chunk, it should end up containing all the elements of that chunk... at least it should exist." Lesson: for any new code that partitions a collection across multiple downstream units — chunks, buckets, shards — the very first test to write is not "does the final output look plausible" but "does every input element show up exactly once in the output, by name." That check is cheap, sits right next to the partitioning logic itself, and would have caught this in minutes instead of hours; a misleading downstream symptom several layers removed is exactly the wrong place to start looking first.

Act XLII: a compiler that compiles itself, and the one register it forgot to save

With the full bundle compiling cleanly, the genuine test of self-hosting was to take the resulting native executable and hand it its own source code, asking it to compile itself a second time, entirely independent of the interpreter or the rustc-backed path that had built it. It segfaulted immediately. Asked to plan the investigation efficiently rather than dive straight into guesswork given how notoriously time-consuming crash debugging can be — "I guess tracking down that segfault has to come next... typically a tricky problem, so develop a strong plan for doing it as efficiently as possible" — the plan settled on real debugger-driven root-causing rather than incremental bisection, made possible by a genuinely useful piece of information volunteered directly: "windbg is at [path]", a real Windows debugger this environment has no command-line equivalent of (no gdb, no cdb). Getting a GUI-packaged, Windows-Store-installed debugger to run non-interactively from a script took a few false starts of its own — including one entirely self-inflicted dead end, where the debugger's own "file not found" error was initially chased as a command-line quoting problem, before it turned out the target executable really had been deleted by an earlier, unrelated cleanup pass and simply needed rebuilding — but the working invocation was found: pass the target and its own arguments as a single joined string, not as separate quoted pieces, plus a short attach-run-log-quit script of debugger commands.

The resulting crash backtrace pointed at the exact same function that backs exec_capture — the one whose stack-alignment pattern had been reused as the known-good template throughout this whole arc's process and networking work. The register holding a saved stack pointer, meant to be restored once a temporary 16-byte-aligned stack frame was no longer needed, had been silently clobbered — by an entirely ordinary call to this program's own heap allocator, sitting in between the save and the restore. That call is just another compiled PatLang function like any other, and this backend's own established rule (documented in the same file, just not applied consistently here) is that a compiled function-to-function call gives no register-preservation guarantee at all — unlike a real Win32 API call, which does. Two such allocator calls, in two separate branches of the same function, had been placed inside that window without the save/restore wrapper the rule already demanded elsewhere. A short Python script audited every stack-pointer-saving window in the entire file against every allocator call site to confirm these were the only two instances of the mistake anywhere, rather than trusting that finding two meant there weren't a third. Both fixed with a simple push-before/pop-after around the vulnerable calls; a fresh full rebuild of the compiler, followed directly by handing it its own source code a second time, completed without a crash — and the resulting output matched, byte for byte, what the interpreter and the rustc-backed path had always produced for the same input. Asked, mid-investigation, whether the whole family of string-comparison dispatch this arc kept running into should eventually become a real hashed lookup table instead of a long comparison chain — "if all else fails, should we be looking at building a hashed name table... to achieve the goal?" — the honest answer is that it's a genuinely good idea worth doing on its own merits, not yet needed to close out this particular bug, and not done in this pass. Lesson: "this call never preserves registers" is easy to write down as a rule once and then quietly forget to apply the next three times a stack-alignment pattern gets copied elsewhere in the same file — an audit that checks every instance mechanically, rather than trusting that the two found by hand were the only two, is worth the extra few minutes it costs.

Act XLIII: real classes, a parallel build, and a fixpoint that turned out not to be one

With one-generation self-hosting proven — patc1.exe compiling itself into a byte-identical patc2.exe — the next question was a genuine architectural one: did the whole ~340-function compiler bundle really need recompiling from scratch every single time, or could "known good" functions be cached the same way any other build system caches unchanged object files? The framing came with a real constraint attached: "it should not be necessary to compile the whole thing at once... Explore a genuine separation of concerns/pragmatic approach to compiling small units and linking them, which might also mean not having to recompile 'known good' (as in, not proven to be broken!) components until they need recompiling by caching the obj files. If this were designed with good OO discipline this should be relatively easy?" Asked to go ahead with a concrete plan, one scoping question turned into a two-part project on its own: real PatLang class/new/send support had never actually been wired into the native backend at all — every use of the feature so far had only ever gone through the interpreter — so building a cache as genuine PatLang objects meant implementing that support first, then the caching feature on top of it. Both landed cleanly: single, non-inheriting classes with real field storage and method dispatch, then an ObjCache/CompileUnit/X64UnitLinker trio doing exactly what the framing asked — a per-function content hash as the cache key, a from-scratch compile only on a genuine miss. A follow-up instinct proved right without much persuading: "I believe, theoretically, we should now be able to run multiple compilations of chunks at the same time... and if we updated the harness to do that and interrupted the current run, we would lose very little because we have cached results from the current run?" — both halves confirmed correct (unique per-unit filenames already prevented clashes, and the cache only ever promotes a file in after a successful assemble), and parallelised the same day.

Then came the sentence that reset every other priority in the room: "Oh actually priority after parallelisation is #31 we must fix that as it is blocking." Issue #31 — patc1_final.exe segfaulting the instant it tried to recompile its own source, the one thing the whole self-hosting effort actually needed to prove — had been open since before this arc's earlier acts, patched around rather than closed. Root-causing it for real, rather than trusting the first plausible-looking fix, took two entirely separate bugs, found in sequence. The first: s[i] string indexing lowers to exactly the same host call as list indexing, and the native implementation of that call unconditionally applied the LIST's 8-byte-per-element memory layout to a STRING too — a byte-level type confusion that had simply never been exercised by anything the compiler's own internals did, until the new caching feature's JSON-based content hashing indexed into a string for the first time. Confirmed with the smallest possible repro: let s="abc"; print(s[1]) printed 256 under the native backend instead of "b". Fixed by checking the value's own family tag before choosing a byte layout, rather than assuming list shape for every indexing operation.

That fix rebuilt cleanly and the fixpoint test was run again — and it still failed, but differently: a genuine stack overflow this time, not a type-confusion crash. WinDbg traced it into a large, ordinary-looking loop, and the second bug turned out to be much more general than the first, and much more quietly dangerous: a bare statement call whose result is never assigned to anything — sb_push(sb, x) used on its own line, never let r = sb_push(...) — leaves its return value sitting on the compiler's own operand stack forever. The interpreter never notices, because its operand stack is just a heap-allocated list that tolerates unbounded growth without complaint. The native backend uses the CPU's real, roughly one-megabyte stack for that exact same purpose — and a large loop containing even one such statement leaks one stack slot per iteration until it runs clean off the end of real, physical stack space. Confirmed directly: a 30,000-iteration loop with two unassigned calls per iteration crashed with a genuine stack overflow; the identical loop with the results assigned first ran cleanly. Every PatLang function already ends with an explicit return, so the leaked value was never actually used for anything — the fix simply discards it, reusing the existing variable-store instruction rather than inventing a new one, in both the Rust-native lowerer and its self-hosted PatLang mirror. Lesson: a design choice that's completely harmless for one execution model (an unbounded heap-backed operand stack) can be a slow-motion crash waiting to happen for another (a fixed-size machine stack) — and the bug can sit invisible for a very long time if nothing in the codebase's own internals happens to trigger the specific pattern until a much later, unrelated feature does.

Act XLIV: a second crash, a proposal politely declined, and the actual cause underneath both

With both bugs fixed, committed, and the regression suite showing no new damage, the natural next step was the real test: not just "does patc1 compile itself once more without crashing," but "does the compiler it produces actually work." It did — patc2.exe correctly compiled and ran a fresh test program, output byte-for-byte identical to every other execution path. Pushed one generation further out of genuine curiosity rather than any requirement — could patc2.exe compile itself into a patc3.exe? — it crashed almost instantly, in a fraction of a second rather than the several minutes a real compile takes. WinDbg again: a write through an address just past the very end of the compiler's own fixed 16-megabyte heap region, which happened to be the last thing reserved in that section of memory — meaning nothing stopped a large enough compile from silently handing out addresses past the end of it and walking straight into unmapped memory.

In the middle of investigating that, a genuinely interesting tangent arrived: a shared conversation proposing a hybrid small-string-optimisation-plus-chunked-rope string representation, prompted by "Given we keep hitting string problems anyway, can we consider moving to this type of hybrid core representation of strings for PatLang?" Read closely and weighed honestly rather than adopted on momentum: none of the three real bugs found this arc were actually about string layout at all — the indexing bug was a tag-dispatch problem a chunked layout would still need the exact same check to solve, and the heap crash was an allocator-robustness problem a chunked design would make measurably worse, since every string append would become a fresh heap allocation instead of one amortized-growth buffer. The proposal's own stated sweet spot — huge, frequently-mid-spliced documents — doesn't match what PatLang's strings actually are in practice: short, append-heavy source chunks and identifiers, exactly what the existing growable-buffer builder already handles well. The recommendation went back plainly rather than being executed reflexively, and the user's own reply drew the line precisely: "It wasn't proposed as a fix for what we are encountering now, but as a possible direction which might be worth considering now. But yet - the heap alloc should grow our heap, and we should definitely not fail silently AND we might want to consider the size first."

That became the real fix: the bump allocator now checks its own usage against the reserved size before handing out an address, failing loudly with a clear diagnostic message and a distinct exit code rather than silently corrupting whatever memory came next — and the reservation itself grew thirty-two-fold, from 16 megabytes to 512, a legitimate and essentially free change on 64-bit Windows specifically because that memory is lazily backed by real RAM only as it's actually touched, not committed up front. Rebuilt, and pushed hard this time rather than stopping at "no longer crashes": patc1 compiled itself into patc2, patc2 compiled itself into patc3, and — for good measure, since three generations proves the fix rather more convincingly than two — patc3 compiled itself into a patc4 as well, each generation independently confirmed to actually work by running the same test program through it. Four generations deep, all agreeing. Lesson: a plausible-sounding architectural idea is worth a fair, specific hearing rather than either instant adoption or instant dismissal — and sometimes the honest answer to "should we rebuild the foundation" is "no, but here's the much smaller, much more targeted fix that actually addresses what broke."

Act XLV: closing the list, and one more bug caught by actually running the code

Two items were left on the backend's own remaining primitive gap list from earlier in this arc — read_line and list_dir — alongside a third, rustc_build, that the native pipeline genuinely has no use for and was explicitly waved off: "So we don't need the rustc_build; the others though should be implemented." Both went in as real WinAPI calls rather than another attempt at shelling out through cmd.exe (an earlier attempt at copy_file/rename_file via the shell this same arc had already run into two separate, genuinely different problems doing exactly that — a documented quoting quirk in how cmd.exe's own /C switch reparses multiple quoted arguments, and a second, unrelated bug in the backend's own captured-output buffer). read_line reads one byte at a time via a direct system call until it hits a newline or end of input; list_dir walks a real directory listing via the Windows file-enumeration API, deliberately split into five narrow primitives with the actual loop — skip . and .., suffix directories with a trailing slash — left as ordinary PatLang, matching this whole arc's established habit of keeping hand-written assembly as narrow as possible and pushing everything else back into the language it's building.

The first test run of both surfaced one more real bug, caught by actually exercising the code rather than trusting it looked right on the page: file names were coming back missing their very last character, and a bare handful of extra bytes' worth of unrelated garbage was leaking into an adjacent result. Both new primitives had allocated a result string exactly as long as its text content, forgetting the small fixed-size header every string in this backend also needs to carry alongside its actual bytes — an eight-byte shortfall that let each new allocation quietly spill one string's worth of leftover data onto its neighbour. A one-line fix once found, and found specifically by running the new code against a real directory and a real piped line of input rather than by re-reading the assembly and assuming it matched what every earlier, already-correct primitive in the same file already did. Regression suite clean, and the fixpoint chain — patc1 through patc4, all four generations, all agreeing — held with every remaining gap now closed but the one deliberately out of scope. Lesson: writing new code that closely mirrors an existing, working pattern is not the same claim as having actually followed that pattern correctly — the eight-byte header this bug forgot is written out, explicitly, in half a dozen other functions in the very same file; the only way to be sure a new one got it right is to run it against real data and look at what comes back.

Lessons from this arc, the short version

"Architecturally clean" can smuggle in a language-choice preference that contradicts a project's own stated direction. If it does, the fix is a visible correction on the record, not a quiet edit.
A crash that "cascades into a wall of unrelated errors" almost always has one root cause near the very start of the generated output, not many separate ones — check the first reported line, not all of them.
Silent-wrong-answer bugs are the dangerous class, not crashes. A negated float that looks like a plausible, slightly-off number is far easier to miss for a long time than an outright failure.
Fixing one bug can make a log honest enough to expose a second, unrelated one sitting right behind it. Don't stop investigating just because the crash you were chasing stopped happening.
A closure-capture bug and a per-function-typing bug can look identical from a garbage number alone. An isolated repro with one variable removed at a time is what tells them apart.
A misgiving that a "cosmetic" bug might actually be the important one is worth following all the way down. A print routine correctly rejecting garbage data is not the same fault as a print routine being broken — and the strongest fix is often hiding one layer further back than the symptom.
Test more broadly than the one case that just got fixed. The very next thing checked after the string-concatenation fix turned up its direct sibling bug, sitting in the same few lines of code.
Investigate a merge before attempting it. A branch feared to be badly out of date turned out to share nearly all of its history already — and the one real difference contained a safety fix worth adopting rather than overwriting.
A brand-new kind of test can find brand-new kinds of bugs in code considered long settled. A benchmark suite built for timing figures and cross-backend confidence found two genuine, previously-invisible bugs — a three-way branch mishandled since whenever it was first written, and a shared utility's own quiet, undocumented assumption — purely by combining language features nothing had happened to combine quite that way before.
"Real implementation now" beats a stub, even for code paths a workload probably never exercises. A stub that never gets called is indistinguishable, by testing, from one that's silently wrong the one time it does.
A partitioning function's first test should check the partition itself, not the pipeline stage three steps downstream. "Does every input element show up exactly once in the output" would have caught a dropped chunk in minutes instead of four hours of chasing a phantom assembler bug.
A misleading error message can point confidently at the wrong layer. NASM's own multi-pass "label changed" diagnostic looked like an optimizer convergence failure; the real fault was a genuinely undefined symbol one call site away, from a chunk silently dropped upstream.
A rule written down once ("this kind of call never preserves registers") still needs enforcing every time the pattern is copied, not just the first time it's stated — and a mechanical audit across every instance beats trusting that the two found by hand were the only two.
A design safe for one execution model can be a slow-motion failure for another. An unbounded heap-backed operand stack tolerates an unassigned statement result forever; a fixed real machine stack does not — the same code, two completely different outcomes, depending only on which stack is underneath it.
"It no longer crashes" and "it works" are different claims — check both, and check them more than once. A compiler that successfully rebuilds itself once is a real milestone; a compiler that can rebuild itself a second, third, and fourth time, each generation independently verified, is the claim that's actually being made.
A plausible architectural proposal deserves a fair, specific hearing — and sometimes the honest answer is still no. Weigh it against the actual bugs on the table, not against how elegant it sounds in isolation, and say so plainly either way.
Copying an existing pattern is not the same as having followed it correctly. The one detail a new implementation is most likely to get wrong is the detail so routine in every other instance that nobody thinks to double-check it — the fix is to run the new code against real data, not to re-read it and nod.