run_test` never receives a START signal — `signalCount: 0` / timeout

Back to board
### OS Windows 11 ### Node.js version 24.16.0. ### AI client Claude Code 2.1.177 / Opus 4.8 ### Plugin / MCP server version 2.7.18 ### Steps to reproduce **Environment** | | | |---|---| | Plugin version | 2.7.18 (current `@latest`) | | AI client | Claude Code 2.1.177 | | Model | Claude Opus 4.8 | | OS | Windows 11 Pro (10.0.26200) | | Tier | PRO | | Install method | `irm .../install.ps1 \| iex` → `npx -y @weppy/roblox-mcp@latest --setup` | **Steps** 1. Connect Studio to the MCP plugin (the command channel connects fine). 2. Confirm the command channel works — e.g. `execute_luau` returning `game.PlaceId` responds instantly. 3. Run a trivial test via `system_info run_test` (timeout 60): ```lua print("__CHANNEL_PING__ server alive seq=1") print("__CHANNEL_PING__ done") ``` This reproduces 100% of the time, in **5 different places**, with **all other Studio plugins disabled**, and after a **fully clean reinstall** of `@latest` (still v2.7.18). See "Actual behavior" for everything already ruled out. --- ### Expected behavior The injected test body runs in the playtest, `__WEPPY_MCP_START__` is recorded, the `print()` lines are captured, `signalCount > 0`, and the test passes (this is exactly how it behaved until 2026-06-13 — 91 passing runs are archived on disk). --- ### Actual behavior Every test times out with `signalCount: 0`. The injected body's output and its `__WEPPY_MCP_START__` signal never appear — only `__WEPPY_MCP_STOP__` warnings at the very end. **Key observations** - ❌ No `__WEPPY_MCP_START__` anywhere — only repeated `__WEPPY_MCP_STOP__` `[warning]` lines at the end. - ❌ The injected `__CHANNEL_PING__` prints never appear — the test body produced no captured output at all. - ✅ In a content-rich place, the game's own server/client logs ARE captured (50+ real log lines during boot) — so `LogService` capture itself works; only the test signal never comes through. - 🔁 The captured block is re-sent repeatedly with the `seq` counter resetting to 1 each cycle — the runner appears to poll the same buffer over and over waiting for a START/DONE signal that never arrives, until it times out. **This is a recent regression, not a longstanding bug** (from 130 archived `test-report.md` files): | When | Result | |---|---| | 2026-05-17 → 2026-06-13 (afternoon) | 91 passed (signalCount ≥ 1) across many places — worked reliably ~4 weeks | | 2026-06-13 ~16:44 | last successful run (signalCount 1, passed) | | 2026-06-13 ~14:14–14:18 | transient cluster of signalCount-0 timeouts, then recovered (~14:20 passed again) | | 2026-06-13 ~16:53 → now | every run times out with signalCount 0 — 10+ consecutive failures across 4 places | The plugin (v2.7.18) was last updated ~2026-06-07 and kept working for ~6 days after that, so the plugin update is **not** the trigger. The permanent break on the evening of 2026-06-13 points to an **external change (most likely a Roblox Studio auto-update / FFlag rollout)** altering playtest/log/injection behavior. **Already ruled out** - [x] Transport / Node — command channel + general log capture both work over the same connection. - [x] Plugin conflict — reproduced with all other Studio plugins disabled. - [x] Place-specific / data corruption — reproduced in 5 unrelated places (incl. a brand-new empty place with a fresh sync folder). - [x] Stale sync state — the fresh-place sync folder did not help. - [x] Corrupt / outdated local install — fully clean reinstall (deleted plugin `.rbxm`, cleared `%LOCALAPPDATA%\npm-cache\_npx` + `npm cache clean --force`, re-ran `install.ps1`); freshly pulled `@latest` loads as v2.7.18 and the failure is unchanged. - [x] `stop` → fresh `play`, full Studio restart, MCP client restart — none fix it. **Hypothesis:** because general `LogService` output is captured but the injected test body's `__WEPPY_MCP_START__` / `print` output is not, the failure is most likely in injecting/running `ServerScriptService.__MCP_TestRunner` in the playtest server DataModel, or in correlating its START signal — not in the log pipe itself. --- ### Logs (if any) ```shell `run_test` result: { "passed": false, "timedOut": true, "durationMs": 61990, "signalCount": 0 } `test-report.md` of a failing run (uniform across all failures): - Status: timed_out - Signal Count: 0 ## Signals - START: missing - FINISHED: missing `test-log.txt` tail — empty place, only plugin lifecycle gets through, repeated ~11× with seq resetting, then only STOP warnings: [seq=1] [info] [LogHandlers] Started log capture [seq=2] [info] [WeppyRobloxMCP] Plugin loaded - v2.7.18 [seq=3] [info] [WorkspaceWatcher] Initialized - watching for changes [seq=4] [info] [WeppyRobloxMCP] Plugin loaded. Use the MCP Panel to connect. [seq=5] [info] (other plugin line; later runs had ALL other plugins disabled — no change) [seq=6] [info] [WeppyRobloxMCP] Command channel connection attempt started... [seq=7] [info] [SelectionMonitor] Started monitoring selection changes ... (the same seq=1..7 block repeats ~11×, seq resets each time) ... [seq=11] [warning] __WEPPY_MCP_STOP__ [seq=12] [warning] __WEPPY_MCP_STOP__ For contrast, in a content-rich place the game's real logs ARE captured (proving log capture works), e.g.: [seq=36] [info] [ReplicaClient]: Initial data received [seq=43] [info] Client successfully loaded in 1.58s [seq=54] [info] [ShopController] Echte Robux-Preise geladen …but the injected `__CHANNEL_PING__` lines and `__WEPPY_MCP_START__` are still absent, and `signalCount` is still 0. Notes (optional context): in the `execute_luau` sandbox `require` is `nil`, so modules can't be compile-/run-checked there — which is exactly why a working `run_test` matters. Separately, this plugin version also crashes Studio on local save while enabled (disabling it lets the save succeed) — possibly the same background sync/capture hook. ```