Viggle Research · 6 min read
We see the world one view at a time, but understand much more than what is visible. Objects remain when we look away. We can imagine moving around them and anticipate how they will move and interact.
Yet many world models learn in pixel space, where the world’s physical structure must be learned indirectly from how it looks.
For four years, we have been building an architecture that learns that structure directly. Today, we introduce JST-2, our physics-native world foundation model. It brings scenes, characters, and motion into a shared 3D world you can enter and explore, with scenes generated and expanded in seconds.
Worlds with scenes, characters, and motion
Scenes, characters, and motion generated with JST-2.
A physics-native architecture
JST, short for Joint Space-and-Time, models the world as persistent atoms in 3D space. Each atom represents a piece of matter or an object part and retains its identity as it moves and deforms. It has an explicit xyz position and learned vectors for its attributes and dynamics. We encode these atoms as Physics-Native Tokens.
We built the full stack for this fundamentally new architecture from the ground up, including its encoder, generator, and decoder, physics-native data synthesis engine, and pre- and post-training pipelines. The same world representation can be inferred from real-world observations or generated from a prompt.
Start with one or more input images of a scene. Tap the images to explore the encoder path.
The JST encoder maps images to Physics-Native Tokens representing persistent atoms in 3D space.
Each persistent atom represents a piece of matter or an object part. Its Physics-Native Token contains:
The JST generator produces Physics-Native Tokens from a multimodal prompt. Tap the card to explore the generator path.
Drag the camera around the room. The JST decoder uses this viewpoint to produce an image of the decoded scene.
The decoder uses motion or force inputs to predict changes in position and attributes. Drag the puck to explore this input.
The JST decoder produces images or 3D Gaussian splats from the tokens. Tap either output to highlight its path.
Everything at once. Drag the camera, push the puck, tap a source or an output.
We founded Viggle in 2022 and spent the first two years developing JST-1. JST-1 validated the architecture on character generation and animation. It powered Viggle’s meme-making app, which has since reached 50M+ users.
We’ve loved seeing people use it to make each other laugh. Thank you for creating with us and sharing the feedback that helps us keep improving the model.
Building on that work, we developed JST-2 from 2024 to 2026. It demonstrates the generality of the JST architecture across scene, character, and motion generation. The representation supports both spatial structure and change over time.
Scene generation at interactive speed
Scenes generated with JST-2, decoded into 3D Gaussian splats. Try making your own
JST-2 generates a 3D scene in about one second, at an inference cost below $0.01. Its feed-forward architecture produces the 3D representation directly, without per-scene optimization.
Explore and expand
JST-2 supports up to 64 input views as a spatial context window. As you expand a world one view at a time, each new view adds to the context for what comes next.
JST-2 decodes its Physics-Native Tokens into 3D Gaussian splats for local browser rendering. Once generated, the 3D scene can be explored repeatedly from different viewpoints without rerunning the model. Model inference is needed when new content is generated.
Modeling the real world
JST-2 reconstructs complete 3D environments from sparse real-world observations, preserving both their structure and appearance. This offers a way to build and expand robot simulation environments from real places, with potential applications in autonomous driving, mapping, and construction.
Upcoming releases will let you use text prompts to guide what appears as you expand a scene. We are also preparing a version with more Gaussian splats per scene for finer visual detail.
Characters with motion built in
JST-2 generates 3D characters from a single image, ready to animate without a separate rigging step. Try making your own
JST-2 generates 3D characters from text or images with motion dynamics built in. The dynamics vectors in their Physics-Native Tokens encode how each part deforms under different motions. These dynamics are learned from data, so characters are ready to animate without separate rigging and skinning steps.
JST-2 can also generate complete characters from cropped or partially occluded images, across a wide range of visual styles.
Human evaluators compared JST-2 with Meshy 7.1 and Tripo H3.1 in blind tests.
JST-2 leads both baselines on character generation, with its strongest results on facial detail. These details help preserve the character’s identity in 3D.
Character generation | vs Meshy 7.1 | vs Tripo H3.1 |
|---|---|---|
| Average Score | 58.4% | 73.2% |
| Clothing quality | 55.7% | 72.1% |
| Texture quality | 56.8% | 70.8% |
| Body quality | 56.8% | 74.2% |
| Face quality | 63.0% | 82.0% |
| Overall Preference | 59.9% | 66.9% |
To evaluate animation readiness, we compared how each model's generated characters move. JST-2 leads both baselines, particularly in hand articulation and how the body and clothing deform during motion.
Animation readiness | vs Meshy 7.1 | vs Tripo H3.1 |
|---|---|---|
| Average Score | 73.8% | 88.9% |
| Joint alignment | 65.6% | 85.9% |
| Rig usability | 72.4% | 86.2% |
| Deformation quality | 77.6% | 93.2% |
| Hand rigging | 82.8% | 89.3% |
| Overall Preference | 70.6% | 89.6% |
Upcoming releases will improve how complex clothing deforms during motion and how hair moves.
Full-body 3D motion from text or video
JST-2 generates full-body 3D motion from text or video. For video inputs, it reconstructs visible movement and uses learned dynamics to infer how unseen parts of the body move.
JST-2 motions can be applied directly to JST-2 characters, without a separate retargeting step. They are also natively compatible with standard 3D character rigs. Together with generated scenes, these characters and motions form the worlds shown above.
Human evaluators assessed both tasks in blind comparisons. All clips were played on the same untextured MetaHuman rig, keeping the comparison focused on motion rather than appearance.
For text-to-motion, JST-2 leads both baselines across the evaluated criteria. The model performs particularly well on hand gestures and dance, following the intent of the prompt while producing natural, continuous movement.
Text-to-motion | vs Cartwheel Swing | vs HY-Motion-1.0 |
|---|---|---|
| Average Score | 74.8% | 80.0% |
| Prompt following | 65.8% | 76.5% |
| Motion quality | 79.8% | 80.2% |
| Foot contact | 73.0% | 80.7% |
| Overall Preference | 80.7% | 82.8% |
JST-2 leads both video-to-motion baselines overall. The reconstructed 3D motion retains the source performance's rhythm and changes of speed, along with fine hand and finger movements.
Video-to-motion | vs Cartwheel Comic 4 | vs DeepMotion Animate 3D |
|---|---|---|
| Average Score | 58.1% | 89.1% |
| Motion accuracy | 56.7% | 90.9% |
| Complex / Fast | 56.0% | 86.2% |
| Motion smoothness | 57.1% | 82.8% |
| Hand tracking | 57.9% | 95.0% |
| Overall Preference | 62.5% | 90.6% |
Upcoming releases will reconstruct a person’s path through the scene from video, alongside their body motion. We will also improve motion accuracy.
JST-3: toward learned physics
In JST-2, character motion is driven by learned dynamics. Scene collisions and object interactions still rely on rule-based physics. We are developing JST-3 to replace those rules with learned dynamics that predict how objects interact and how those interactions change the world.
Our goal is to build an experience engine for physical AI, where agents learn by acting in simulated worlds and observing the consequences.