LUCID / NEURAL WORLDS 009 — RESEARCH PROTOTYPE
2026-11-05

Play: https://neural-flappy.vercel.app/world.html
Earlier construction-program engine: /studio.html
Source: /neural-engine-source.zip
Measured results: /world/report.json

WHY THIS ARCHITECTURE

Google's Genie 4 describes a combination of visual generative models and reasoning
models for coherent interactive worlds. Training that capability from scratch is
outside this experiment. This prototype instead uses pretrained visual knowledge
to write a compact 3D memory that can be rendered cheaply between edits.

Words -> Qwen language director -> SDXL image synthesis -> SHARP reconstruction
      -> persistent anisotropic Gaussian field -> interactive GPU projection.

This avoids teaching a new network what every object looks like. It also avoids
running a diffusion model for every displayed frame. The same stored field is
reprojected for each camera pose. Changing the view therefore gives real parallax,
occlusion and stable appearance around the reference camera, not a video playback.

NEURAL AND CONVENTIONAL PARTS

Neural: open-vocabulary language generation; image generation; prediction of 3D
Gaussian positions, colors, opacity and anisotropic covariance from an image.
Conventional: field packing, camera controls, depth sorting, projection and alpha
compositing, UI, masks and scene history. Subtle foliage movement and ambient sound
are analytic/procedural. The live representation is a radial-basis field, not a
trained dense MLP renderer. No new model training was performed for version 009.
The previous fully hand-encoded and fine-tuned neural-engine experiments remain
unchanged and accessible at the existing URLs.

WHAT YOU CAN DO

Drag to look. WASD/arrow keys or the touch thumbstick moves the camera. The
movement volume is intentionally near the source view: +/-0.45 m horizontally,
-0.2 to +0.72 m forward; yaw +/-0.38 radians; pitch +/-0.22 radians.

Transform: describe a change. A language model combines the instruction with the
current scene description; diffusion synthesizes a new image; SHARP reconstructs
a new field. Shapes can change and details can drift.

New world: describe an arbitrary environment. It is generated from text without
a predefined object library. Model knowledge and prompt following are limited.

Change here: hold a spot in the view for half a second, then describe what should
appear. A local inpainting mask preserves the surrounding rendered image. The
result is reconstructed in 3D, so geometry may shift slightly even outside the mask.

Explore onward: uses the current rendered view as the visual seed for another
place. This is generative continuation, not spatially registered map expansion.
It does not guarantee that unseen surfaces or topology are correct.

Return: restores the preceding field and camera pose. Fields are immutable once
generated. Two decoded fields are cached to make nearby returns cheaper.

Four saved worlds are hosted statically. Their imagery is pregenerated, clearly
separate from live text generation. Choosing a saved world is not a model test.

OPTIMIZATION

SHARP predicts 1,179,648 Gaussians in two depth layers. The mobile export samples
each layer at stride 3 (~130k Gaussians) and increases covariance support to cover
the lower sampling density. Position remains float32; color/opacity use 8 bits;
the six symmetric covariance entries use float16. Each Gaussian is 28 bytes on
disk, compared with the initial 64-byte float32 record. Gzip transfers are about
3.3–3.4 MB per saved world; reconstruction weights never go to the phone.
The optional Detail button loads the stride-2 field, about 7.3–7.5 MB and 294k
Gaussians, for sharper saved worlds. It retains the current camera. Live generated
worlds currently use the smaller field only. Detail is opt-in, not a hidden download.

The browser decompresses, decodes and sorts in a worker. Sorting is skipped when
orientation is unchanged; translation does not alter back-to-front order. The
renderer uses one instanced draw, a float texture, projected anisotropic ellipses,
and premultiplied alpha. Pixel ratio is capped at 1.5 and width at 1440 pixels.
Hidden tabs stop rendering work. The canvas does not retain its drawing buffer;
captures explicitly render a fresh frame before readback.

The language director runs on the desktop GPU only during interpretation, then
moves back to CPU memory to release room for diffusion and reconstruction.
Diffusion uses 4 SDXL-Turbo scheduler steps, zero classifier-free guidance, and
FP16 weights. Image-to-image uses the corresponding fraction of those steps.
SHARP inference uses autocast FP16. Covariance is transformed directly rather
than running ~1.18 million CPU singular-value decompositions.
Unused CUDA activation buffers are released after each job so the workstation
does not retain the peak VRAM reservation while idle.

VALIDATION AND LIMITS

See report.json for actual timings. Desktop draw times are about 0.2–0.3 ms at
1280x720 on an RTX 5080 for the saved mobile fields. These are GPU-only offscreen
measurements, excluding sorting, browser overhead and download. They are NOT
phone frame rates. The browser displays its own measured animation frame rate.

The exact vertex/fragment shaders were compiled and rendered with native OpenGL
on the GPU. Origin, left, right and forward views of all four scenes were inspected.
Codec, camera basis, depth order, UI load/return/prompt/error flow and all earlier
engine tests pass. The public HTTP pipeline is tested from prompt through LLM,
diffusion, reconstruction, CORS, gzip download and binary validation. Browser
automation is disabled in this workspace, so actual Safari/Chrome interaction
and phone performance have not been verified. DOM tests use mocked graphics;
the native GPU tests cover actual rendering, not browser integration.

This is a hybrid research demo, not Genie 4, a general physics simulator, an
unbounded 360-degree world, or a continuously generated 24 fps video model.
Disoccluded surfaces can stretch or blur. Single-image metric geometry is only
an estimate. Generated objects may be wrong or missing. Continuations may change
structure. Sound is procedural, not generated from the world's semantics.

HOSTING

The saved worlds and frontend are on Vercel. Live generation uses the user's
RTX 5080 through a temporary Cloudflare HTTPS tunnel and requires that computer,
server and tunnel to remain running. No paid generation API is used. The public
research endpoint accepts one job at a time, limits prompt length and view size,
enforces a short cooldown and caps each server session at 64 generated worlds.
It exposes only generated assets and narrow job endpoints, not the workspace.
If the tunnel stops, the four saved worlds still work. Restarting the tunnel
changes its URL and requires updating world/connection.json and redeploying.

REPRODUCTION

Python 3.12, PyTorch 2.11.0+cu128, RTX 5080 16 GB, 64 GB system RAM.
Source entry points: scripts/world/generate.py, reconstruct.py, director.py,
server.py, render.py, verify-server.py. Browser entry: src/world-app.js.

Dependencies installed in artifacts/world-deps: diffusers 0.35.2,
huggingface-hub 1.33.0, timm 1.0.31, gsplat 1.5.3, plyfile 1.1.5 and pillow-heif.
Global dependencies include transformers 5.5.4, Pillow, NumPy, FastAPI, uvicorn.
Native GPU validation uses moderngl from artifacts/gpu-deps.

Clone https://github.com/apple-aiml-research/ml-sharp into artifacts/ml-sharp.
Download its official sharp_2572gikvuh.pt checkpoint into artifacts/world/sharp.pt.
Cache the pinned SDXL and Qwen files from their official Hugging Face repositories.
Run python scripts/world/server.py. Server binds only 127.0.0.1:8787.
Run npm start to serve static files locally at 127.0.0.1:5173.
Run npm test; python scripts/world/render.py garden lite; and
python scripts/world/verify-server.py for independent checks.
Downloaded pretrained checkpoints and vendor dependencies are not in the archive.

MODELS, LICENSES, SOURCES

Qwen/Qwen2.5-1.5B-Instruct, Apache 2.0
revision 989aa7980e4cf806f80c7fef2b1adb7bc71aa306
https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct

stabilityai/sdxl-turbo, Stability AI Community License (included)
revision 71153311d3dbb46851df1931d3ca6e939de83304
https://huggingface.co/stabilityai/sdxl-turbo

Apple SHARP, Apple Machine Learning Research Model License (included)
https://github.com/apple-aiml-research/ml-sharp
https://ml-site.cdn-apple.com/models/sharp/sharp_2572gikvuh.pt
https://arxiv.org/abs/2512.10685
SHARP is research-only. This experiment is for noncommercial scientific
evaluation; it is not presented as a commercially deployable engine.

Genie research inspiration, with no affiliation or use of Google's model:
https://deepmind.google/models/genie/
https://deepmind.google/blog/genie-4-a-new-generation-of-world-simulation/

ART PROVENANCE

The initial garden was created with the built-in image-generation tool in
generation mode. The other three saved worlds were synthesized locally with
SDXL-Turbo (seed 24), then reconstructed with SHARP. Live examples are genuinely
model-generated from the submitted prompt, with random per-job seeds saved in
their metadata. The initial garden prompt follows:

Generate one photorealistic wide 16:9 first-person landscape image as a reconstruction
test asset for an explorable world. Scene: the entrance to a forgotten water garden
in a vast lush temperate valley, a beautifully weathered circular stone arch ten meters
ahead, a clear shallow turquoise stream crossing between worn stone stepping paths,
ferns and flowering moss, vertical waterfalls far in the gorge, pale mountains in
atmospheric distance. Camera at human eye height, 22mm rectilinear wide angle,
looking slightly down along an open walkable foreground path. Strong distinct depth
layers: detailed wet stone and ferns close to the camera, the immense arch and stream
at middle distance, waterfalls and mountains beyond. Cinematic late-afternoon sunlight,
warm shafts through light mist, rich natural green, luminous clean water, extraordinary
real-world texture and subtle imperfect photographic detail. It should feel like a wondrous
real place someone can step into, not stylized videogame blocks or an illustration.
Clear sharp photograph across the whole scene. No people, no text, no UI, no watermark,
no collage. Full-bleed landscape image.
