Fix five multiplayer sync bugs, add automated two-instance test

Adds test/multiPlayerTest.tscn plus a driver that runs the kitchen flow
across two game instances: grab/drop, dirt station, sink washing, hob
cooking, counter combining and plating. Runs headless (run_mp_test.ps1),
in two visible windows (run_mp_test_windowed.ps1), or by hand with
keyboard controls (play_mp_test.ps1). 108 checks, exits non-zero on
failure.

After every step both peers snapshot every item's position and rendered
state and the server diffs them. Targeted assertions only look at the
thing a step touched, which misses desyncs elsewhere - that audit is
what caught the last bug below.

Bugs found and fixed:

- net_pickable: apply_held_state() only wrote `enabled` in its
  non-authority branch, so once a client grabbed an item every other
  peer set enabled=false and regaining authority never restored it. The
  server could then never pick that item up again, and a station would
  "snap" it (emitting has_picked_up, so a plate still got marked dirty)
  while pick_up() bailed out on the disabled item - leaving the zone
  holding an item with no grab driver.

- network_manager: station gating only happened in the spawn path, so
  stations baked into a scene file kept running their snap zones on
  clients and grabbed items straight out of the local hand. Added
  gate_existing_stations().

- network_manager: despawn_item() only freed the server's copy. Items
  baked into a scene aren't tracked by the MultiplayerSpawner, so
  consuming one left a ghost on every client, which then blocked the
  station it sat in and got grabbed instead of its replacement.

- network_manager: the snap-into-station decision read the server's own
  copy of the item position, but the reliable release RPC routinely
  overtakes the synchronizer's unordered position updates - so it acted
  on a stale position and teleported items back into the station they
  had just been carried away from. The releasing peer now sends its
  final transform and the server adopts it first.

- container: contained_ids.append()/erase() mutate the array in place,
  which never fires the setter that rebuilds the plate's visuals. The
  peer that put food on a plate was the only peer that never redrew it;
  remote peers looked right because the synchronizer assigns there.

Also null-guards XRServer.get_tracker() in the vendored xr-tools hand
grab point, which threw on every successful grab without an XR runtime,
and adds multiplayer_world.populate_from_layout so debug scenes can bake
their own content instead of spawning the whole kitchen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
algodoogle
2026-07-26 02:00:50 +01:00
parent 33ef81306d
commit 23ec41c1d6
12 changed files with 1503 additions and 12 deletions
File diff suppressed because it is too large Load Diff
+1
View File
@@ -0,0 +1 @@
uid://biu4qr3nr1eny
+47
View File
@@ -0,0 +1,47 @@
# Shared helper for placing the two game windows side by side.
#
# Godot's --position flag is ignored on this setup (the window lands at x=-7
# whatever you pass), so the windows are moved with the Win32 API after they
# come up. Dot-source this file to get Move-GameWindow.
if (-not ('MpWin' -as [type])) {
Add-Type @"
using System;
using System.Runtime.InteropServices;
public class MpWin {
[DllImport("user32.dll")] public static extern bool SetWindowPos(IntPtr h, IntPtr after, int x, int y, int cx, int cy, uint flags);
[DllImport("user32.dll")] public static extern bool GetWindowRect(IntPtr h, out RECT r);
[DllImport("user32.dll")] public static extern bool SetProcessDPIAware();
public struct RECT { public int Left, Top, Right, Bottom; }
}
"@
}
# Godot's windows are DPI aware; PowerShell's process is not by default, so
# GetWindowRect/SetWindowPos would otherwise be talking in virtualised
# coordinates and the windows land nowhere near where we asked.
[void][MpWin]::SetProcessDPIAware()
# Moves a just-launched game window to (x, y) and returns its width, so the
# caller can place the next window immediately to its right. Never resizes:
# changing the window size independently of Godot's --resolution distorts the
# rendered aspect ratio.
function Move-GameWindow($proc, [int]$x, [int]$y) {
for ($i = 0; $i -lt 60 -and $proc.MainWindowHandle -eq 0; $i++) {
Start-Sleep -Milliseconds 250
$proc.Refresh()
}
if ($proc.MainWindowHandle -eq 0) {
Write-Host " (window never appeared; leaving it where it is)"
return 620
}
# The handle shows up before Godot has finished sizing/positioning the
# window - moving it too early gets overwritten by Godot's own setup.
Start-Sleep -Milliseconds 2500
$h = $proc.MainWindowHandle
# SWP_NOSIZE (0x1) | SWP_NOZORDER (0x4)
[void][MpWin]::SetWindowPos($h, [IntPtr]::Zero, $x, $y, 0, 0, 0x0005)
$r = New-Object MpWin+RECT
[void][MpWin]::GetWindowRect($h, [ref]$r)
return [int]($r.Right - $r.Left)
}
+112
View File
@@ -0,0 +1,112 @@
[gd_scene format=3 uid="uid://bodj8op527o2c"]
[ext_resource type="Script" uid="uid://d1cefhe3yyyds" path="res://Scenes/multiplayer_world.gd" id="1_6uucx"]
[ext_resource type="PackedScene" uid="uid://j5s5wus7tuhb" path="res://XROrigin.tscn" id="2_j7vd1"]
[ext_resource type="Material" uid="uid://cmia50cfqxxo4" path="res://Textures/dev_material_3d.tres" id="3_ssbaf"]
[ext_resource type="Sky" uid="uid://c1abtpwhdm2d5" path="res://Textures/sky/NightSkyHDRI009_16K.tres" id="4_081u3"]
[ext_resource type="PackedScene" uid="uid://j7caslh27nor" path="res://Stations/Hob.tscn" id="5_t1fa7"]
[ext_resource type="PackedScene" uid="uid://dvrk268s7gkxh" path="res://Stations/sink.tscn" id="6_6jhmh"]
[ext_resource type="PackedScene" uid="uid://bp3v1jl8pctro" path="res://Containers/plate.tscn" id="7_bowes"]
[ext_resource type="PackedScene" uid="uid://cnjwtnhwh0i8q" path="res://Stations/dirt_station.tscn" id="8_pvl84"]
[ext_resource type="Script" uid="uid://biu4qr3nr1eny" path="res://test/mp_test_driver.gd" id="9_mptst"]
[ext_resource type="PackedScene" uid="uid://dpot5qie20vf6" path="res://Items/burger.tscn" id="10_51k0c"]
[ext_resource type="PackedScene" uid="uid://c0lknik4noobs" path="res://Items/BurgerBuns.tscn" id="11_psgbv"]
[ext_resource type="PackedScene" uid="uid://efaec6ymgabo" path="res://Stations/Counter.tscn" id="12_j5uvh"]
[sub_resource type="BoxShape3D" id="BoxShape3D_vlqg6"]
size = Vector3(15, 0.1, 15)
[sub_resource type="BoxMesh" id="BoxMesh_24d3s"]
material = ExtResource("3_ssbaf")
size = Vector3(15, 0.1, 15)
[sub_resource type="Environment" id="Environment_bvwq1"]
background_mode = 2
sky = ExtResource("4_081u3")
reflected_light_source = 2
ssr_enabled = true
ssao_enabled = true
ssil_enabled = true
sdfgi_enabled = true
[sub_resource type="BoxShape3D" id="BoxShape3D_arao0"]
size = Vector3(15, 20, 0.1)
[node name="Main" type="Node3D" unique_id=1312265607]
script = ExtResource("1_6uucx")
populate_from_layout = false
[node name="XROrigin3D" parent="." unique_id=2055526621 groups=["local_xr_origin"] instance=ExtResource("2_j7vd1")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0.36171648, 0, 0.81835127)
[node name="DirectionalLight3D" type="DirectionalLight3D" parent="." unique_id=1376644384]
transform = Transform3D(-0.8660254, -0.43301278, 0.24999999, 0.3022632, -0.05510214, 0.9516306, -0.39829266, 0.8997021, 0.17860365, 0, 0, 0)
[node name="Floor" type="StaticBody3D" parent="." unique_id=122279514]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, -0.05, 0)
[node name="CollisionShape3D" type="CollisionShape3D" parent="Floor" unique_id=958841648]
shape = SubResource("BoxShape3D_vlqg6")
[node name="MeshInstance3D" type="MeshInstance3D" parent="Floor/CollisionShape3D" unique_id=1939997056]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0.00018164515, 0.0006352663, -0.002532661)
mesh = SubResource("BoxMesh_24d3s")
[node name="WorldEnvironment" type="WorldEnvironment" parent="." unique_id=1177077250]
environment = SubResource("Environment_bvwq1")
[node name="WorldContent" type="Node3D" parent="." unique_id=291550153]
[node name="Players" type="Node3D" parent="." unique_id=1595463693]
[node name="ItemsSpawner" type="MultiplayerSpawner" parent="." unique_id=627119248]
spawn_path = NodePath("../WorldContent")
[node name="PlayersSpawner" type="MultiplayerSpawner" parent="." unique_id=106645565]
_spawnable_scenes = PackedStringArray("res://Player/net_player.tscn")
spawn_path = NodePath("../Players")
[node name="StaticBody3D" type="StaticBody3D" parent="." unique_id=4404969]
[node name="CollisionShape3D" type="CollisionShape3D" parent="StaticBody3D" unique_id=998476672]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 9.197384, 7.589958)
shape = SubResource("BoxShape3D_arao0")
[node name="CollisionShape3D2" type="CollisionShape3D" parent="StaticBody3D" unique_id=340303902]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 9.197384, -7.509142)
shape = SubResource("BoxShape3D_arao0")
[node name="CollisionShape3D3" type="CollisionShape3D" parent="StaticBody3D" unique_id=1193703037]
transform = Transform3D(-4.371139e-08, 0, -1, 0, 1, 0, 1, 0, -4.371139e-08, -7.438612, 9.197384, -0.018813243)
shape = SubResource("BoxShape3D_arao0")
[node name="CollisionShape3D4" type="CollisionShape3D" parent="StaticBody3D" unique_id=1431367494]
transform = Transform3D(-4.371139e-08, 0, -1, 0, 1, 0, 1, 0, -4.371139e-08, 7.502467, 9.197384, -0.018813023)
shape = SubResource("BoxShape3D_arao0")
[node name="Hob" parent="." unique_id=1687971542 instance=ExtResource("5_t1fa7")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0.5, 0)
[node name="Sink" parent="." unique_id=2055277359 instance=ExtResource("6_6jhmh")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0.5, 0)
[node name="Plate" parent="." unique_id=190487773 instance=ExtResource("7_bowes")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 1.4050963, 1.1702834, 0.049627244)
[node name="DirtStation" parent="." unique_id=160842153 instance=ExtResource("8_pvl84")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 2, 0.5, 0)
[node name="TestDriver" type="Node" parent="." unique_id=1002011928]
script = ExtResource("9_mptst")
[node name="raw_burger" parent="." unique_id=1675596942 instance=ExtResource("10_51k0c")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0.28692234, 1.0149999, 0.3505687)
[node name="BurgerBuns" parent="." unique_id=1088240294 instance=ExtResource("11_psgbv")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, -0.40037438, 1.0094403, 0.34120744)
[node name="Counter" parent="." unique_id=1487893288 instance=ExtResource("12_j5uvh")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, -1, 0.5, 0)
[node name="Counter2" parent="." instance=ExtResource("12_j5uvh")]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, -2, 0.5, 0)
+61
View File
@@ -0,0 +1,61 @@
# Opens two windows on the multiplayer test scene in MANUAL mode, so you can
# drive the plate / dirt-station flow yourself and watch both peers.
#
# powershell -File test\play_mp_test.ps1
#
# No scripted sequence runs. Click a window to focus it, then:
#
# 1 grab the plate 2 drop it
# 3 carry it to the dirt zone 4 drop it at the dirt zone
# 5 check it snapped + went dirty
# 6 run the whole automatic sequence (server window only)
# 0 dump the current state of everything
# C toggle between the fixed debug camera and the XR rig camera
#
# Keys act on whichever window has focus, so you can grab on the CLIENT and
# watch the SERVER window follow. Each window opens on a fixed camera looking
# at the test area, overlays its own step log, and writes it to
# logs\mptest_server.log / logs\mptest_client.log.
#
# The typical repro: on the CLIENT press 1, 2 (grab and drop), then on the
# SERVER press 1, 2. Then on the CLIENT press 1, 3, 4 and press 5 on both.
param(
[string]$Godot = "C:\Users\Aqua Aurora\Desktop\Godot_v4.7-stable_win64.exe",
# Pass -Solo to open a single window with no networking, to compare the
# same steps against single-player behaviour.
[switch]$Solo
)
$ErrorActionPreference = "Stop"
$proj = Split-Path -Parent $PSScriptRoot
$scene = "res://test/multiPlayerTest.tscn"
. (Join-Path $PSScriptRoot "mp_window_layout.ps1")
function Start-Instance($extraArgs) {
$a = "--xr-mode off --resolution 900x600 --path `"$proj`" $scene"
if ($extraArgs) { $a += " -- $($extraArgs -join ' ')" }
return Start-Process -FilePath $Godot -ArgumentList $a -PassThru
}
if ($Solo) {
Write-Host "Opening a single offline window (no networking)."
$null = Move-GameWindow (Start-Instance $null) 300 100
return
}
Write-Host "Opening SERVER window (left)..."
$w = Move-GameWindow (Start-Instance @("--server")) 20 60
Start-Sleep -Seconds 5
Write-Host "Opening CLIENT window (right)..."
$null = Move-GameWindow (Start-Instance @("--join", "127.0.0.1")) (20 + $w + 12) 60
Write-Host ""
Write-Host "Both windows are up. Click one to focus it, then press:"
Write-Host " 1 grab 2 drop 3 carry-to-dirt 4 drop-at-dirt 5 verify 0 dump"
Write-Host " 6 runs the whole automatic sequence (server window only) C toggles the camera"
Write-Host ""
Write-Host "Step logs: $(Join-Path $proj 'logs\mptest_server.log')"
Write-Host " $(Join-Path $proj 'logs\mptest_client.log')"
Write-Host "Close the windows when you're done (Esc quits)."
+60
View File
@@ -0,0 +1,60 @@
# Runs the headless two-instance multiplayer test (test/multiPlayerTest.tscn).
#
# powershell -File test\run_mp_test.ps1
#
# Starts a server instance and a client instance of the game with --xr-mode off
# (SteamVR's OpenXR runtime crashes a headless process), lets test/mp_test_driver.gd
# drive the scripted plate/dirt-station sequence, then prints both logs.
# Exits non-zero if any check failed.
param(
[string]$Godot = "C:\Users\Aqua Aurora\Desktop\Godot_v4.7-stable_win64.exe",
[int]$TimeoutSec = 120
)
$ErrorActionPreference = "Stop"
$proj = Split-Path -Parent $PSScriptRoot
$scene = "res://test/multiPlayerTest.tscn"
$serverLog = Join-Path $proj "logs/mptest_server.log"
$clientLog = Join-Path $proj "logs/mptest_client.log"
foreach ($f in @($serverLog, $clientLog)) { if (Test-Path $f) { Remove-Item $f -Force } }
function Start-Instance($extraArgs) {
# Single argument string with the project path quoted: Start-Process does
# not quote array elements, so the space in the path would split it.
$a = "--headless --xr-mode off --path `"$proj`" $scene -- $($extraArgs -join ' ')"
$p = Start-Process -FilePath $Godot -ArgumentList $a -PassThru -NoNewWindow
# Touching .Handle caches it, which is what makes .ExitCode readable later;
# without this it comes back empty even after the process has exited.
$null = $p.Handle
return $p
}
Write-Host "Starting server..."
$server = Start-Instance @("--server", "--mptest")
Start-Sleep -Seconds 4
Write-Host "Starting client..."
$client = Start-Instance @("--join", "127.0.0.1", "--mptest")
$deadline = (Get-Date).AddSeconds($TimeoutSec)
while ((Get-Date) -lt $deadline -and -not $server.HasExited) { Start-Sleep -Milliseconds 500 }
foreach ($p in @($server, $client)) {
if (-not $p.HasExited) { Write-Host "Killing pid $($p.Id) (still running)"; $p.Kill() }
}
Start-Sleep -Milliseconds 500
foreach ($pair in @(@("SERVER", $serverLog), @("CLIENT", $clientLog))) {
Write-Host ""
Write-Host "======================== $($pair[0]) ========================"
if (Test-Path $pair[1]) { Get-Content $pair[1] -Encoding UTF8 } else { Write-Host "(no log written)" }
}
# ExitCode is only populated on the process object after a WaitForExit() call,
# even when HasExited is already true - without this it reads back empty.
$server.WaitForExit(2000) | Out-Null
$code = if ($server.HasExited) { $server.ExitCode } else { 1 }
Write-Host ""
Write-Host "server exit code: $code"
exit $code
+69
View File
@@ -0,0 +1,69 @@
# Runs the two-instance multiplayer test in two VISIBLE windows, side by side,
# pausing between steps so you can watch what happens on each peer.
#
# powershell -File test\run_mp_test_windowed.ps1
#
# Same test as run_mp_test.ps1 (headless); this one is for watching it. Each
# window shows an on-screen overlay of the step log for that peer.
#
# To drive it yourself instead, see test\play_mp_test.ps1 (manual mode).
param(
[string]$Godot = "C:\Users\Aqua Aurora\Desktop\Godot_v4.7-stable_win64.exe",
# Seconds to pause between steps, and to hold the final state on screen.
[double]$Pause = 1.5,
[double]$Hold = 20,
[int]$TimeoutSec = 300
)
$ErrorActionPreference = "Stop"
$proj = Split-Path -Parent $PSScriptRoot
$scene = "res://test/multiPlayerTest.tscn"
$serverLog = Join-Path $proj "logs/mptest_server.log"
$clientLog = Join-Path $proj "logs/mptest_client.log"
foreach ($f in @($serverLog, $clientLog)) { if (Test-Path $f) { Remove-Item $f -Force } }
. (Join-Path $PSScriptRoot "mp_window_layout.ps1")
function Start-Instance($extraArgs) {
# --xr-mode off keeps it on the desktop (SteamVR's OpenXR runtime otherwise
# takes over, and two instances can't share a headset anyway).
$a = "--xr-mode off --resolution 900x600 --path `"$proj`" $scene -- " +
"$($extraArgs -join ' ') --mptest --mptest-pause $Pause --mptest-hold $Hold"
$p = Start-Process -FilePath $Godot -ArgumentList $a -PassThru
# Touching .Handle caches it, which is what makes .ExitCode readable later.
$null = $p.Handle
return $p
}
Write-Host "Starting SERVER window (left)..."
$server = Start-Instance @("--server")
$w = Move-GameWindow $server 20 60
Start-Sleep -Seconds 5
Write-Host "Starting CLIENT window (right)..."
$client = Start-Instance @("--join", "127.0.0.1")
$null = Move-GameWindow $client (20 + $w + 12) 60
Write-Host "Watch the two windows (each has a fixed camera on the test area)."
Write-Host "Step logs: $serverLog"
Write-Host " $clientLog"
$deadline = (Get-Date).AddSeconds($TimeoutSec)
while ((Get-Date) -lt $deadline -and -not $server.HasExited) { Start-Sleep -Milliseconds 500 }
foreach ($p in @($server, $client)) {
if (-not $p.HasExited) { Write-Host "Killing pid $($p.Id) (still running)"; $p.Kill() }
}
Start-Sleep -Milliseconds 500
foreach ($pair in @(@("SERVER", $serverLog), @("CLIENT", $clientLog))) {
Write-Host ""
Write-Host "======================== $($pair[0]) ========================"
if (Test-Path $pair[1]) { Get-Content $pair[1] -Encoding UTF8 } else { Write-Host "(no log written)" }
}
$server.WaitForExit(2000) | Out-Null
$code = if ($server.HasExited) { $server.ExitCode } else { 1 }
Write-Host ""
Write-Host "server exit code: $code"
exit $code