Changes made during a playtest are never synced, and not caught up after stop
Back to board**Versions:** `@weppy/roblox-mcp` 2.17.10 (adapter, bridge and Studio plugin identical). Windows 11 Pro 10.0.26200, Roblox Studio 0.740.19.7400003, Node v24.16.0, MCP client Claude Code 2.1.283.
**Sync policy:** v2 preset `studio_first`, all directions forward, apply manual, `reverseDetectorMode: snapshot-scanner`, `fileWatcherActive: true`, state `syncing` throughout (never `idle`, never a reconnect during the measurements).
**Place:** ~4,490 instances, ~290 scripts. All probes below target one small, long-mirrored ModuleScript in ServerStorage (~500 bytes) and one Folder next to it; nothing else was touched during the measurement windows.
## 1. What happens
While Studio is in Play mode, `execute_luau` still writes into the plugin's Edit DataModel (`RunService:IsRunning()` is `false` from the plugin's point of view, and reading the property back confirms the write). But a change made in that window — a `.Source` write, an attribute, a newly created instance — never reaches the mirror: not during the playtest, and not after it is stopped. `manage_sync history` has no entry for it, `lastIncrementalSync` does not move, and the plugin log shows no `[SyncManager] Sending …` for it. The plugin *does* notice the change: for an instance created during Play the log has `[SyncWatcher] Instance added: …`, but no `Sending` follows, and stopping the playtest triggers no reconciliation.
Two details narrow it down:
- A later Edit-mode write to the **same** script with **different** content is sent normally and carries the lost change along. Re-assigning the **identical** source in Edit mode sends nothing. So the watcher's "last known" state is updated during Play; only the send is suppressed, and the stop does not flush what was suppressed.
- Attributes and new instances created during Play are not brought along by a later unrelated write; they stay missing until something changes on that very instance again.
It does not matter how the playtest is started: Roblox's built-in Studio MCP (`start_stop_play`), weppy's own `manage_studio play_start`, and F5 by hand in Studio all behave the same.
Practical effect for us: a fix written during a playtest looked committed — our git snapshot reads the mirror, our mirror-freshness check was green, the file's timestamp was recent — but the mirror still had the old code. We noticed only because a grep for the new line came back empty.
## 2. Measurements (2026-09-28, times UTC, plugin log `seq` numbers from `manage_logs`)
Each probe appends one comment line to the script's `.Source` (`s.Source = s.Source .. "\n-- probe X\n"`) via `execute_luau`, or sets an attribute / creates a Folder+ModuleScript, then polls the mirrored file every 3 s.
| # | what | Studio state during the write | plugin log | arrived in mirror |
|---|---|---|---|---|
| C | `.Source` + line | Edit mode | `Sending 2 changes: scriptSourceChanged, propertyChanged` (seq 926, 15:18:21Z) | **yes, < 3 s** |
| D | `.Source` + line | Play, started by Roblox Studio MCP `start_stop_play` | nothing | **no** — 60 s during play, 90 s after stop |
| F | `.Source` + line (different content) | Edit mode, after D | `Sending 2 changes` (seq 934, 15:22:40Z) | **yes, < 3 s, and D is now in the file** |
| E | `.Source` + line | Play, started by weppy `play_start` | no attributable line (one `Sending 2 changes` at 15:25:29Z, seq 945, coincides with `play_stop` and carries no path) | **no** — 75 s after `play_stop` |
| G | `.Source` + line | Play (Roblox Studio MCP) | nothing | **no** — 45 s after stop |
| G-attr | `SetAttribute` on a mirrored Folder | Play (same window as G) | nothing | **no** (no `.props.json` created) |
| G-new | new Folder + ModuleScript under ServerStorage | Play (same window as G) | `[SyncWatcher] Instance added: …Folder` / `…ModuleScript` (seq 953/954, 15:33:51Z) — **no `Sending`** | **no** — still absent 6 min later |
| ident | re-assign the **identical** `.Source` (which contains G) | Edit mode | nothing | **no** (30 s) — the file stays at the pre-G content |
| H | `.Source` + line (different content) | Edit mode | `Sending 2 changes` (seq 955, 15:35:30Z) | **yes, < 3 s, and G is now in the file** |
| control-attr / control-new | `SetAttribute` + new Folder/ModuleScript | Edit mode | `Instance added` ×2, `Sending 2 changes: instanceAdded, instanceAdded`, `Sending 1 changes: propertyChanged` (seq 956–959, 15:36:16–18Z) | **yes, < 3 s** (`.props.json` appears with both attributes; folder appears) |
| E2 | `.Source` + line | Play, weppy `play_start` (repeat of E with clean log) | **no `Sending` at all** after seq 959 | **no** — 60 s after `play_stop` |
| cleanup | remove probe lines / folders / attributes | Edit mode | `Sending 4 changes: instanceRemoved ×4`, `Sending 1 changes: propertyChanged`, `Sending 2 changes` (seq 977–981, 15:39:22–27Z) | yes, 3 s |
| F5 | `.Source` + line | Play, started **by hand in Studio (F5)** and stopped by hand | **no `Sending` at all** after seq 981 | **no** — 120 s during play, 90 s after the manual stop |
Reading of the table: Edit-mode changes are sent within a second (C, F, H, controls); every change made during Play is dropped (D, E, E2, G, G-attr, G-new, F5), regardless of how the playtest was started — Roblox's Studio MCP, weppy's `play_start`, or F5 by hand; the watcher still observes during Play (G-new has `Instance added` lines), the manager does not send; nothing is sent on stop; and a later *different* write to the same script re-sends the whole source (F, H), while an *identical* re-assignment does not (`ident`) — consistent with a last-known-state cache that was updated during Play.
## 3. Reproduction
1. Sync on, state `syncing`. Pick any mirrored ModuleScript.
2. Start a playtest (`manage_studio play_start`, or Roblox's Studio MCP `start_stop_play`).
3. `execute_luau`: append a comment line to the module's `.Source`. The call succeeds; reading `.Source` back shows the line. Optionally also `SetAttribute` on a mirrored instance and create a Folder.
4. Stop the playtest. Wait. The mirrored file, `.props.json` and folder never appear; `manage_sync history` shows nothing.
5. Append another (different) line in Edit mode → the file updates within seconds and contains both lines. The attribute and the folder are still missing.
## 4. History on our side (why we think this has been with us for a while)
- **2026-07-27, 2.11.1:** the plugin stopped sync at every playtest start and re-ran a Full Sync afterwards (2–4 min for ~4,100 instances). At that time the stop produced `WsTransport:31 handleResponse → "cannot spawn non-suspended coroutine with arguments"` in the plugin. A full re-sync after every playtest would have covered any change dropped during Play — so the gap described here could not be observed then.
- **2026-07-31:** one script's mirror copy stayed stale after a change and was carried into git by hand. Not measured against Play state at the time; consistent with this report.
- **2026-08-04, 2.12.0:** first day we logged "eight playtests with sync running and not a single new error" — i.e. sync now survives playtests. From then on, a change made during Play has no full re-sync behind it that would pick it up.
- **2026-08-26 → 2026-09-27:** we built and repeatedly used a bypass that hashes live sources and pulls only the differing scripts straight from Studio into git, because the mirror could not be trusted after test sessions (reasons were mixed: bridge OOM during Full Sync, invalid UTF-8, sibling-suffix duplicates — all reported separately).
- **2026-09-26:** a client script existed in the mirror only as `X~1`/`X~2` copies with stale content after edits; pulled from live.
- **2026-09-28:** a two-line fix written during a playtest (via `execute_luau`, then re-assigned with identical content after stop, then re-written with weppy `edit_replace` on the same line — all three left the mirror untouched) was found missing only by a grep for the new line. That case is exactly the `ident` row above and triggered the measurements.
Related reports already filed: sibling-suffix duplicates (2026-08-17), sync init chunk timeout, place folder archived mid-sync, EPERM rename race, bridge heap OOM during Full Sync (2026-09-28). This report is independent of those: the bridge stayed alive and `syncing` throughout.
## 5. Things we checked that are NOT the cause (negative findings)
- Not a bridge death or reconnect: `system_info connection` unchanged, `manage_sync status_current_place` `syncing` with the same `activeClientId` before and after every probe.
- Not the write itself: every write is confirmed by reading the property back, and the same write in Edit mode syncs in < 3 s.
- Not specific to the native Studio MCP: weppy's own `play_start` shows the same loss (E, E2).
- Not a silent dedupe of identical content only: G-attr and G-new have nothing to dedupe against and are dropped as well.
- Not specific to MCP-started playtests either: a playtest started by hand with F5 and stopped by hand in Studio loses the change the same way (row F5).
## 6. Related observation (sibling-suffix class, probably the report from 2026-08-17)
Four ModuleScripts imported the same session via `manage_assets import_rbxm` into a new Folder behaved differently: **every** `.Source` write to one of them — during Play *and* in Edit mode afterwards — produced a new set of `X~N` sibling folders in the mirror with the *pre-edit* content (`ModuleA~2..~4`, `ModuleB~2..~4`, `ModuleC~2..~3` for one folder that has exactly one instance of each name in Studio), preceded by `[SyncManager] Forward restore: sent 4 Studio values to restore local files` / `… sent 9 …`. Even the Edit-mode control folder of §2 arrived as `Module~2`. Long-mirrored scripts do not show this; a later Edit-mode write updates them in place. Mentioned here only because the two effects overlapped in one session and could be confused for each other.
## 7. Questions
- Is sending intentionally paused while Studio is in Play mode (to keep play-time state out of the mirror)? If so, a reconciliation on stop — compare the watcher's last-known state (which is already updated) against what was last sent — would close the gap; today the updated state is the reason a later identical write sends nothing.
- Could `manage_sync status_current_place` expose "sending paused (Play mode)" and a `pendingChanges` count, so a client can defer writes or verify afterwards?
- Could `[SyncManager] Sending … changes` include the instance paths? The lines currently carry counts and kinds only, which made attributing seq 945 impossible.