LIVING 011 — prompt a playable neural game
November 6, 2026.

PLAYABLE LOOP
play.html adds first-person movement, jumping, sliding box collisions, pickups,
switches, chasing hazards, pulse stun, health, invulnerability, goals and restart.
Those game rules compile to sparse affine/ReLU weights in the same recurrence
as the existing motion engine. The wrapper reads controls, camera and HUD state.
The renderer is still hybrid: learned fields/materials plus fixed ray traversal,
texture interpolation and lighting feature construction. It is not an all-neural
video world model. Collision proxies are coarse axis-aligned boxes.

A small pretrained Qwen model describes either a complete level or an edit.
The compiler assigns identities and derives collection counts for whole levels.
The first attempted command-only planner failed on duplicate IDs and deleted
objects. A whole-level grammar fixed bookkeeping; required duration and explicit
generate-vs-reuse alternatives fixed two additional semantic errors. Three final
trial prompts satisfy their requested counts, duration and novel-shape creation;
all three can be won via neural controls. This is a small evaluated set, not a
claim of arbitrary behavioral generalization. Supported goals: collect, reach,
survive. New mechanics still require new neural circuits/training data.

PROPERTY EDITS
An explicit language-selected mode constrains property edits separately from
new geometry and complete levels. This corrected unnecessary resculpting during
a resize. A neural radius seed also starts orbiting from an exactly stationary
state; the generic inference runtime performs that transition. Two actual model tests preserve asset identities and take 2.24–2.98s
to plan; the resize edit reaches the screen in 5.3s. A new two-object robot
quest plans in 8.63s. Model intent decisions can
still be imperfect. Full generated programs remain available for inspection.

PLAY WHILE APPEARANCE REFINES
A validated game opens with existing neural fields as provisional appearances.
The pretrained visual prior creates requested novel objects, and their compact
field weights replace the provisional visuals without resetting player state.
In actual Chrome, a two-object pineapple quest planned in 11.55s, was observed
playable during object creation, and finished downloading/displaying in 28.9s.
The player moved during generation, retained progress, collected the generated
pineapple and reached the exit. New field calibration: 140 steps, vertex MAE
0.09571 -> 0.00777. This is per-object fitting, not held-out renderer learning.
The saved pineapple quest packs only its two active fields: 13.75MB -> 3.20MB
compressed, with bit-exact retained weights. It is playable from a static host.
Full ten-object planning trials took 26.5–27.4s. These are workstation timings,
not latency guarantees. New text/geometry requires the connected workstation;
gameplay, saved games and five-weight motion training work in the browser.

PERFORMANCE AND VERIFICATION
Starter game: 13 actors, 3,609 recurrent parameters, 9.84ms compilation and
0.01223ms per simulation step across 6,000 steps on this workstation. Rendering
uses adaptive pixel budgets and portable WebGL2 fields; phone hardware varies.
81 JavaScript tests pass, including independent movement/collision oracles,
three generated-level victories and state retention through asynchronous edits.
Exact GLSL ES 3.00 shaders compile on the GPU. Chrome verifies desktop movement,
390x844 touch controls, prompt-to-preview-to-refinement, pickup and victory.
Source: scripts/living/game_program.py, src/living-game.js, src/play-app.js.
Reproduce: npm test; npm run test:living-shaders;
python scripts/living/check_games.py; python scripts/living/evaluate_games.py.
Local gameplay backend: python scripts/living/server.py --port 8790.

PREVIOUS EXPERIMENT / 010
LIVING 010 — a trainable neural engine with a generative asset prior
November 6, 2026. Research prototype, with working source and measured failures.

010.2 PROMPT PROGRESS
Prompt submission immediately shows a progress card and elapsed timer. It stays
in progress through generation, streaming asset download, decoding and display;
the final time includes all those stages. Failed connections can reconnect on
the next submission. Errors preserve the prompt. A Chrome UI test generated and
displayed a new treasure chest in 10.6 seconds total: 8.37 seconds generating and
2.2 seconds connecting/loading/displaying. This is one measured request, not a
latency guarantee. Browser access became available for this verification.

010.1 SHADER FIX
The initial release used "sample", a reserved GLSL ES word, as a shadow-shader
variable. Native desktop GLSL checks missed this, so the browser renderer failed
to start. Renamed it to sceneSample. The regression check now compiles and links
the exact GLSL ES 3.00 sources for all three field variants, and confirms that
the original shader fails. Native frame verification now uses those exact ES
sources too. Run npm run test:living-shaders with the GPU test dependencies.

WHAT CHANGED
009 reconstructed a whole image as a mostly static 3D scene. 010 restores a
persistent object state and an actual recurrent neural update at every tick.
Text edits change that world. A new object receives queryable 3D neural weights,
then participates in the same live simulation as hand-initialized objects.

THE ARCHITECTURE
Text -> pretrained Qwen3-4B -> validated world instructions
New object description -> SDXL-Turbo -> foreground mask -> TripoSR neural field
Neural field observations -> compact feature-grid weights + gradient calibration
Actions + previous state -> sparse affine/ReLU recurrent engine -> next state
Camera + state -> local neural SDF/color fields -> trained neural material -> RGB

The simulation compiler hand-encodes integration, recurrent motors, wind, coarse
contacts and bounce constraints into the original generic ReLU network runtime.
Five force-law weights are learned; the complete recurrence is represented as
weights, but we do not claim every one of those weights has been trained.
The object field is a tensor-product tent-basis feature grid with a shared
4->8->4 ReLU decoder. A 7->24->24->3 network learns surface appearance.
These representations admit gradient training and can represent newly supplied
shapes; they are not a lookup table of supported object names.

Conventional wrapper operations remain: language-program validation and graph
compilation, camera rays, ray traversal, interpolation, normal/light features,
GPU scheduling and optimization. The density-to-distance conversion is also
conventional. This is a hybrid neural engine, not an all-neural implementation
of every graphics operation. We do not present a trained Genie replacement.

WHY USE PRETRAINED OBJECT PRIORS?
This avoids learning arbitrary visual rendering from scratch. The pretrained
models supply an object prior; the engine supplies persistence, controllability,
3D viewpoints and a trainable transition rule. There is no per-frame image
generation. Generated backs can be inaccurate; thin structures can disappear.
The dragon is one generated example, not a special keyword/classifier output.
The selected LLM is pretrained, not our fine-tuned geometry adapter. No gradient
currently passes from the renderer through discrete instructions into the LLM.

EXPERIMENTS AND RESULTS
1. Dynamics: equal 8-step training losses, 1,000 Adam steps; evaluate at 2,000
   steps. Dense 3,126-parameter recurrent model: held-out position RMSE 0.10918,
   out-of-distribution RMSE 0.63418. Structured two-weight model: 0.12292/0.62111.
   Hand-initialized structure plus five learned weights: 0.00507/0.00693.
   A least-squares baseline with the exact known basis and velocity labels
   scored 8.67e-8. That baseline wins when those assumptions are available.
2. On-device training: six trajectories train five weights, two are held out.
   800 gradient steps reduce held-out state RMSE from 0.64774 to 0.00337.
   This is real optimization running in a browser worker, not a timed animation.
3. Shape learning: dense coordinate MLP near-surface SDF MAE 0.01437 / IoU 0.86855;
   feature-grid decoder 0.00344 / 0.96270. Near-surface RGB MAE 1.34/255.
   Direct sampled-grid baseline: 0.00340 / 0.96288, also slightly better here.
   Increased spatial coverage and lattice calibration fixed visible field noise.
4. Joint image-only training: one known object, known cameras/actions, 18
   synthetic RGB frames. Learn motion weights and color from a common image loss.
   Third-camera/unseen-time image MSE: 0.0077016 -> 1.04e-9; position RMSE:
   0.22787 -> 0.0000282. No state labels enter the training loss. Some nonlinear
   coefficients are unidentifiable because the short motion never excites them.
   This is a controlled differentiable-rendering proof, not arbitrary-video
   learning. Its soft training renderer differs from the deployment ray marcher.
5. Language-to-geometry experiments: Qwen2.5 1.5B/3B, Qwen3 4B, grammar constraints,
   relational parts, separate object planning, and three adapter-training runs.
   One adapter's loss fell 0.37026 -> 0.003565, but its unfamiliar shapes remained
   poor. A separate canonical-shape adapter fell 0.94652 -> 0.26833 and visibly
   overfit. Rejected. Valid JSON and weak semantic checks overstated quality;
   actual neural renders are in language-geometry.jpg and prior-geometry.jpg.
6. Selected per-object prior: a fresh red clockwork submarine request completed
   text -> new neural asset -> persistent world in 17.78 seconds. A subsequent
   motion/lighting/ground edit took 2.97 seconds, reusing the field weights.
   These are individual workstation measurements, not latency guarantees.

PERFORMANCE AND CHECKS
Original 8-asset observatory: 1.59 MB compressed field download. Generated dragon
world: 4 unique assets / 6 actors / 6.13 MB compressed, 63^3 features per asset.
FP16 weights, one atlas, batched GPU queries, fixed 60 Hz neural simulation,
adaptive pixel budget, incremental asset downloads, preserved shader programs.
The atlas stays within WebGL2's minimum 256^3 texture capability. At 63^3 the
limit is 16 unique assets; more assets fall back to 32^3, up to 64 unique assets.
The live world supports up to 32 actors; contacts scale quadratically.
Desktop native OpenGL execution checks actual generated shader code and weights.
1,024 independent CPU/GPU field queries agree within 0.00133 for the generated
world. RTX 5080 renders 960x600 in approximately 0.42-0.45 GPU ms in that test.
This excludes browser overhead and is NOT a phone benchmark. Browser automation
was disabled in this session. UI flows use jsdom with the real simulation and
training; GPU rendering was verified separately using native OpenGL.

TRY IT
living.html?world=dragon opens the generated world. Drag/pinch to explore, tap
objects to push, and type an edit. Inside the model -> Train here runs local
gradient training. Initial/trained geometry comparison is in the observatory
(home button). Save downloads the world, state and trained weights as JSON.
Text generation depends on the connected workstation and temporary tunnel.
The saved/static worlds and local training continue to work without that server.
Imported saves currently support local interaction/training but not text edits.
Motion modules are still, spring, orbit and bounce. There is no character rig,
arbitrary learned action repertoire, terrain system or full rigid-body physics.
Whole objects move and deform; the dragon's wings are not independently rigged.
Coarse AABB contacts are optional; the generated dragon scene disables them.

REPRODUCE
npm install; npm run dev
Python 3.12 with CUDA PyTorch; see living/requirements.txt. Then:
python scripts/living/setup.py       (downloads several GB of pinned models)
python scripts/living/server.py      (127.0.0.1:8788)
Set living/connection.json to that endpoint for local development.
python scripts/living/study.py
python scripts/living/fields.py
python scripts/living/material.py
python scripts/living/joint_inverse.py
python scripts/living/check_compiler.py
npm test
node scripts/living/fixtures.mjs; python scripts/living/render.py
node scripts/living/verify-generated.mjs
python scripts/living/render.py artifacts/living/gpu-generated
The shipped neural weights allow offline play without the training environment.
The source ZIP includes engine code, experiments, tests, reports and weights;
large third-party pretrained checkpoints are fetched by setup, not redistributed.

NEXT RESEARCH STEP
Train a recurrent residual on longer multi-object interaction sequences and
distill behavior programs into additional neural modules. Use rendered rollout
and interaction rewards to tune the text planner. Keep exact world memory and
the renderer prior fixed initially. Generalization must be assessed on genuinely
new objects/actions and long rollouts, not training loss or JSON validity.

PRIMARY REFERENCES / MODEL ATTRIBUTION
Interaction Networks: https://arxiv.org/abs/1612.00222
Differentiable simulation: https://arxiv.org/abs/1905.10706
Efficient feature-grid neural fields: https://nvlabs.github.io/instant-ngp/
Qwen3-4B: https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507 (Apache 2.0)
TripoSR: https://github.com/VAST-AI-Research/TripoSR (MIT; TRIPOSR-LICENSE.txt)
SDXL-Turbo: https://huggingface.co/stabilityai/sdxl-turbo (SDXL-LICENSE.txt)
Foreground prior: U2NetP, https://github.com/xuebinqin/U-2-Net (Apache 2.0)
Pinned revisions and checksums are in scripts/living/setup.py and object_prior.py.
