Bridge dies with "JavaScript heap out of memory" during Full Sync: every sync-history flush re-reads and re-parses ALL history files
Back to board## ▼ FIELD 1 — Steps to reproduce
| | |
|---|---|
| weppy | `@weppy/roblox-mcp` **2.17.14**. Bridge (`--bridge-runtime`), adapter and Studio plugin are identical (`system_info connection`) |
| OS / runtime | Windows 11 Pro 10.0.26200, Node v24.16.0, 16 GB RAM |
| Studio / client | Roblox Studio 0.741.19.7411056, MCP client Claude Code 2.1.289, bridge started via `npx -y @weppy/roblox-mcp@latest` |
| Heap limit | `NODE_OPTIONS=--max-old-space-size=8192` in the MCP server env. Before we set it, the bridge died at ~4 GB instead |
| Dashboard settings | defaults (`ENABLE_LOCAL_HISTORY=true`, `ENABLE_CONTEXT_CAPTURE=true`), except Open Cloud upload and script write validation, which are on |
| Sync | preset `studio_first`, all directions forward, apply manual |
1. Use weppy on a place for a while. Ours had **30 files / 277 MB** in `observability/tools/history/` plus ~60 MB of sync histories across 10 place folders. As E1 shows, a smaller tool history alone does not prevent the climb.
2. Open the place (~5,000 instances) and start a Full Sync from the plugin. **Do nothing else.** There are no MCP calls and no manual work in Studio.
3. Watch the working set of the `node … index.js --bridge-runtime` process.
On our machine this kills the bridge in roughly every second attempt. It happened in **4 of 6** measured runs on one place, and in 1 of 1 on a second, unrelated place.
---
## ▼ FIELD 2 — Expected behavior
The Full Sync completes. Memory use should depend on the size of the place being synced, not on how much tool-call history has piled up on disk over the past weeks.
---
## ▼ FIELD 3 — Actual behavior
The bridge's memory climbs without recovering until V8 kills the process:
```
[18496:…] 536443 ms: Mark-Compact (reduce) 8173.6 (8215.7) -> 8172.9 (8186.4) MB, …
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
```
The plugin then shows only `Full sync pre-check failed. Retry after the MCP server is available.`, and `manage_sync` reports a plain `idle`. Nothing tells the user that the bridge process died.
### Same place, same state, same 40 minutes: two runs complete, two die
Working set in GB, maximum per 20 s window, measured from `Full sync started` with a probe every 2 s:
```mermaid
xychart-beta
title "Bridge working set during Full Sync (GB) — A1 completed, A4 died"
x-axis "seconds after Full sync started" [0, 20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 260, 280]
y-axis "GB" 0 --> 8.5
line [3.36, 3.23, 2.48, 2.21, 2.31, 2.42, 2.50, 2.70, 2.33, 2.43, 2.10, 2.41, 2.40, 2.43, 2.21]
line [2.53, 2.27, 2.66, 2.42, 2.67, 2.16, 2.48, 3.75, 5.52, 5.73, 7.25, 7.91, 8.08, 8.33, 8.34]
```
| Run | Place | Activity during sync | Result | Peak |
|---|---|---|---|---|
| A1 | Place A (~5,000 instances) | none | completed (288 s) | 3.4 GB |
| A2b | Place A, same state | none | completed (342 s) | 3.1 GB |
| A3 | Place A, same state | 4 screenshots via a second MCP server | **died** (~288 s) | 8.3 GB |
| A4 | Place A, same state | **none** | **died** (~285 s) | 8.3 GB |
| C1 | **Place B**: unrelated, ~17,600 instances, tool history ≈ 0 MB | **none** | **died, but only after ~820 s** | 8.1 GB |
| P1 | Place A, with heap profiler attached | none | **died** (~320 s) | 8.2 GB heap |
| E1 | Place A, tool history moved out (277 → 15 MB), profiler attached | none | sync **completed** (253 s), but the heap kept climbing and peaked **right after** completion at **8,044 MB** (limit ~8,190), then drained to 1.4 GB. The bridge survived **by ~150 MB** | 8.0 GB heap |
`bridge.log` since 2026-09-03: 64 × `Full sync started`, 32 × `completed`, **26 heap OOM deaths**. They happened on every version from 2.17.10 to 2.17.14, and 25 of the 26 fell inside a Full Sync.
---
## ▼ FIELD 4 — Root cause (measured with a heap sampling profile of a dying run)
We attached the V8 sampling heap profiler to the running bridge (`process._debugProcess` → Inspector → `HeapProfiler.startSampling`). Then we took a profile every 15 s until the process died (run P1).
**Out of ~8,050 MB of live sampled allocations at the last profile, 7,417 MB come from one function:**
| live MB | function (minified, `dist/index.js` 2.17.14) |
|---:|---|
| **7,417** | **`fH` @ index.js:1129:9023** |
| 159 | `APe` @ index.js:1131:3745 |
| 110 | `PPe` @ index.js:1131:2161 |
| 105 | `split` (native) |
| 90 | `join` (native) |
| 73 | `qPe` @ index.js:1131:10297 |
| < 32 each | everything else |
The heap grew steadily from the start of the sync (heapUsed in MB, one profile every 15 s):
```mermaid
xychart-beta
title "P1: heapUsed (MB) after Full sync started"
x-axis "seconds" [0, 15, 31, 46, 62, 77, 93, 109, 125, 141, 157, 175, 194, 212, 230, 250, 269, 285, 301, 319]
y-axis "MB" 0 --> 8500
line [30, 427, 905, 1704, 2584, 3418, 4092, 4773, 5359, 6114, 6643, 6997, 7215, 7494, 7648, 7511, 7807, 7895, 7969, 8162]
```
### The call chain
```mermaid
flowchart TD
A["Full Sync writes instances<br/>(sync writer)"] --> B["sync-history flush<br/>(at 50 entries or every 1 s)"]
B --> C["onHistoryFlushCallback<br/>registered in yJ"]
C --> D["Cd(t, () => vJ(t, placeId))<br/>fire-and-forget: no await,<br/>no coalescing, no lock"]
D --> E["vJ → xs(t, co) → BPe"]
E --> F["$Pe: read EVERY *.jsonl in<br/>observability/tools/history/<br/>of the current place<br/>(ours: 30 files, 277 MB)"]
E --> G["jPe: read observability/sync/history.jsonl<br/>of EVERY place folder<br/>(ours: 10 files, 60 MB)"]
F --> H["fH(text): split on newline,<br/>JSON.parse every line<br/>→ whole file as objects"]
G --> H
H --> I["next flush arrives while the<br/>previous load is still running<br/>→ loads pile up → heap limit"]
```
The relevant code, verbatim from `dist/index.js` (2.17.14; minified names):
```js
// the parser — 7.4 GB of the retained heap is allocated here
function fH(t){let e=[];for(let n of t.split(`\n`)){let r=n.trim();if(!(!r||r.startsWith("#")))try{e.push(JSON.parse(r))}catch{}}return e}
// registered as the flush callback of every sync writer
n.setOnHistoryFlushCallback(()=>{Cd(t,()=>vJ(t,i),{source:"sync-history",placeId:i,placeName:e.placeName??null})})
// Cd only starts the promise and attaches a catch — nothing waits for it, nothing merges calls
function Cd(t,e,n){let r;try{r=e()}catch(i){r=Promise.reject(i)}r.catch(i=>{ /* log + diagnostics */ }).catch(()=>{})}
// vJ reloads ALL sessions on every flush
async function vJ(t,e){let n=await xs(t,co),r=el(n,{placeId:e}); …}
// xs → BPe: tool history of the current place + sync history of every place
async function BPe(t,e){ … a=await $Pe(i,e),o=await jPe(t.config.dataDir), … }
```
### How often this runs per Full Sync (numbers from your code + our sync history)
- `flushHistory()` runs when the buffer reaches **50 entries** (`historyBuffer.length>=DH`, `DH=50`) **or every 1 s** (`flushTimer=setInterval(…, yCe)`, `yCe=1e3`). After every *successful* flush it calls `onHistoryFlushCallback`.
- On Place A a Full Sync writes **~11–15 instances per second**. Measured from our `history.jsonl`:
| run | writes | seconds with ≥1 write | writes/s |
|---|---|---|---|
| A2b | 4,351 | 297 | 14.6 |
| A3 | 2,800 | 246 | 11.4 |
| A4 | 3,450 | 241 | 14.3 |
That is below the 50-entry threshold, so almost every flush is triggered by the 1 s timer.
- ⇒ **Our estimate (derived, not counted directly):** roughly one flush, and therefore one full reload, per second of sync. That is about **240–300 reloads per Full Sync**, and 87 at the very least (4,351 / 50).
- Each reload reads **~337 MB of JSONL** on Place A (277 MB tool history + 60 MB sync histories) and parses all of it. Over one Full Sync that adds up to tens of GB of `JSON.parse`, *if* the reloads ran to completion one after another. Any reload that is still running when the next one starts keeps its own parsed copy alive.
- `flushHistory()` itself also re-reads and rewrites the place's whole sync `history.jsonl` (~12 MB) on every flush (`readFile … split … join … atomicWrite`). The native `split`/`join` (195 MB live in the profile) probably come from there and from `fH`.
- A flush that fails with EPERM does not call the callback. During A2b / A3 / A4 there were 13 / 4 / 8 such failures, which is small against ~250 flushes.
### Why this explains everything we saw
- **"Sometimes it completes, sometimes it dies."** Whether a run dies depends on whether each full reload finishes before the next flush arrives. If it does, the memory is freed and the curve stays a sawtooth (A1, A2b). If it does not, the loads overlap and every one of them holds its own parsed copy, so memory climbs monotonically (A3, A4, P1). *(Interpretation: the profile proves where the memory sits; the overlap is inferred from `Cd` not awaiting.)*
- **Place B died much later** (~820 s vs. ~285 s). Its own tool history is almost empty, so each reload only parses the 60 MB of sync histories. *(Fits, but after E1 we no longer think file size is the main factor; Place B also has a long phase of ~450 s in `ReplicatedStorage` before the climb starts.)*
- **The peak can come after the sync.** In E1 the heap was highest seconds *after* `Full sync completed`, which matches a backlog of reloads that are still being worked off.
- **It got worse over the weeks.** `observability/tools/history/` only grows, so every flush has more to re-parse.
- **The EPERM rename failures** we reported earlier (`rename history.jsonl.tmp.* → history.jsonl`) fit this too. `jPe` opens exactly that `history.jsonl` for reading on every flush, and on Windows a rename onto a file that is open fails with EPERM. *(Plausible link, not measured separately.)*
---
## ▼ FIELD 5 — Already ruled out / what we measured before filing
- **Not place-specific.** An unrelated second place dies the same way (C1).
- **Not caused by activity during the sync.** A4, C1 and P1 had no MCP calls and no manual Studio work. weppy itself rejects mutations during a Full Sync with `SYNC_BUSY`.
- **Not a large script folder or a lot of source in the place.** Place B has 2.3 MB of source, and the dying runs on Place A died before our large backup folder was even written.
- **No duplicate writes.** Every `write` entry in the sync history of A2b, A3, A4 and C1 has a distinct `artifactPath`, e.g. 3,450 of 3,450 in A4. The counter was checked with a positive control in which two identical lines count as one.
- **Not invalid UTF-8.** All ~520 scripts of Place A were checked with a calibrated detector: 0 invalid.
- **Not mixed versions or non-default history settings.** Both history switches are at their shipped default (`true`).
- **Not our heap limit.** `--max-old-space-size=8192` only moves the point of death; before we set it, the bridge died at ~4 GB.
- **Mitigation test E1, so deleting old history on our side is NOT a fix:** we moved the 25 older tool-history files (263 MB) out of `observability/tools/history/`, which leaves 15 MB, and repeated the same sync with the profiler attached.
- The sync completed.
- But the heap climbed at **the same rate as in the dying run P1** (~800 MB per 15 s: P1 400 → 881 → 1,678 → 2,548 MB; E1 799 → 1,619 → 2,506 → 3,308 MB).
- The heap peaked at 8,044 MB right after `Full sync completed`, still in state `syncing`, and only then drained.
- Again **6,621 MB** of the live sample sat in `fH`.
- We verified in the code that `getHistoryDir` resolves to `observability/tools/history/` of the current place, so the move did shrink each reload.
- *Our interpretation:* the pile-up grows at the speed the CPU can `JSON.parse`, not at a rate set by file size. Smaller files only let the queued reloads finish (and free memory) sooner after the writes stop. In E1 that happened just before the limit; in P1 it did not.
---
## ▼ How we measured (so you can repeat it on any machine)
**Memory curve:** a PowerShell loop that every 2 s logs the `WorkingSet64` / `PrivateMemorySize64` of the `node … --bridge-runtime` process. The process is found by `CommandLine`. The loop also notes when the PID changes, which means the bridge died and restarted.
**Heap sampling profile of the live bridge (no restart, no config change):** a small Node script with roughly this core:
```js
process._debugProcess(bridgePid); // opens the inspector in the running bridge (works on Windows)
const [{ webSocketDebuggerUrl }] = await (await fetch('http://127.0.0.1:9229/json/list')).json();
// over that WebSocket (CDP):
// HeapProfiler.enable
// HeapProfiler.startSampling { samplingInterval: 65536 }
// every 15 s: HeapProfiler.getSamplingProfile → save JSON
// Runtime.getHeapUsage → heapUsed
// then flatten the profile tree to selfSize per (functionName @ url:line:col) and diff the first vs. the last profile
```
Attaching costs nothing measurable at idle (heapUsed stayed at 23 MB for 2.5 min before the sync started). The profiles contain function names, script URLs with line/column, and sizes. They contain no heap contents.
**Sync timeline:** `observability/sync/history.jsonl` has one `write` entry per instance with a timestamp, which gives "where was the sync at second X". This matters because `GET /sync/status` carries no progress object.
## ▼ Attachments available on request
Memory CSVs at 2 s resolution, the heap sampling profiles of P1 and E1, and a table of all 26 deaths. Because this issue is public, we would share these privately, with local paths and project names removed first.