Runway Solaris: When Software Interfaces Are Generated Frame by Frame

Published August 31, 2026 · Technology On the Net

On August 31, 2026, Runway introduced Solaris, a system it calls the first Interface World Model. The pitch is blunt: instead of loading apps and websites that were coded months earlier, generate the interface itself—frame by frame—as someone uses it. No intermediate HTML, no frozen component tree, no “design then implement” handoff. The pixels on screen are the application.

That claim sits at the intersection of video generation, agent training, and product design. It is also early. Solaris is not a public product yet; Runway is collecting early-access requests and working with partners. Still, the technical story is concrete enough to unpack, and the practical questions for builders are already clear. This post covers what Solaris is, why the idea matters, how Runway says it works, what it cannot do yet, and what readers should watch as the space moves.

Image credit: Runway Solaris announcement imagery (source: runwayml.com)

Figure 1. Conceptual contrast: fixed multi-screen UIs versus a continuously generated interactive interface.

What problem Solaris is trying to solve

Modern software still runs on a translation step. Designers and models can produce near-final looking screens. Those screens do not run. Someone—or another model—must convert the visual into code, widgets, routes, and event handlers. That intermediate representation is useful for reliability and accessibility, but it is also a compression of what was possible. Behaviors that were not anticipated during implementation simply do not exist.

Runway’s argument is that this translation discards information and locks interaction into predefined paths. Multimodal language models can try to reconstruct an interface from a screenshot, but reconstruction quality drops as visual complexity rises. Natural images and image-heavy pages lose the most detail when forced through language. Solaris sidesteps that loop: it treats the visual state as the interface state and updates it by generating the next frames under user control.

In other words, Solaris is not “AI that writes React.” It is AI that renders interaction the way a real-time world model renders a scene—except the scene is a storefront, a tutorial, a product demo, or some other software surface.

What an Interface World Model is

Runway frames Interface World Models as systems that must do two jobs that used to live in different stacks:

• Know something about the user’s intent, products, and task (the job usually done by search, assistants, or business logic).
• Respond in real time with rich visuals (the job usually done by browsers, game engines, or interactive simulators).

A search box answers with text or a static card. A game engine can be immersive but knows nothing about your catalog. Solaris tries to be both: a model that understands what you asked for and continuously synthesizes an interactive world around that request.

Runway highlights three product-facing properties:

1. Entirely visual. The rendered image is the application. In a clothing demo, you might drag a shirt onto a reference image of yourself without a separate try-on pipeline implemented as a feature.
2. Alive. Because frames keep streaming, the environment can keep moving—lighting shifts, materials respond—rather than waiting for the next click on a dead page.
3. Open-ended. Interactions are not limited to the handlers a developer registered. The same scene can support different behaviors depending on how the driving models interpret input.

That last point is the sharp edge. Flexibility is powerful for demos and exploratory experiences. It is also where trust, correctness, and accessibility get harder.

Image credit: Solaris interaction example from Runway (source: runwayml.com)

Figure 2. High-level flow: user input → language model reasoning → Solaris real-time rendering.

How Solaris works at a useful level

Solaris builds on Runway’s Gen-4.5 video model and follows the company’s broader world-model line (including GWM-1). The adaptations that matter for interfaces are interaction conditioning and real-time response.

Interaction as conditioning

User actions—clicks, drags, and related input—are treated like other conditioning signals (text prompts, reference images). The model sees only past interactions, never future ones, and learns associations between actions and visual outcomes. That is how Runway claims the system can respond to manipulation without every behavior being hand-coded as a UI event.

Making video generation interactive

Standard video diffusion is too slow for UI. Refining a whole clip over dozens of denoising steps can take seconds to minutes. Runway describes a three-stage conversion into a real-time engine:

1. Train for autoregressive frames so each frame depends only on what came before.
2. Distill many-step denoising into a few steps.
3. Train the fast model on its own outputs so quality stays stable across longer sessions.

The target is interactive-speed generation with visual quality holding around 720p. Runway did not publish a precise latency number in the announcement, but the design goal is clear: stay under the roughly half-second threshold where interaction stops feeling interactive.

Split brain: LLM reasons, Solaris renders

Solaris does not do everything alone. A language model interprets requests, decides whether to mutate the current scene or transition to a new one, defines ambient behaviors, and produces prompts that steer rendering. Solaris then generates the visible frames. Reasoning and rendering are separated on purpose: one system decides what should happen next; the other decides how that looks and responds.

Starting state matters. You give the system an initial brand environment or product scene. Frames stream. As the user acts, those actions fold into subsequent frames. Prompts can redefine what a click means in context—Runway’s “redefining the mouse” idea—so selecting one object can turn later clicks into a brush that transfers that object’s material or style.

Why this matters beyond a flashy demo

There are three practical reasons this is worth attention even before public launch.

1. Product surfaces may stop being fixed catalogs of apps. If an OS-like layer can generate a useful interface for the task at hand, the unit of interaction is less “open the shopping app” and more “show me what I need right now.” Runway explicitly sketches storefronts that keep brand identity while reshaping layout, materials, and recommendations per visitor.

2. Agent training needs mutable interfaces. Computer-use agents trained on coded UIs often overfit layouts. A hotel booking site that shuffled controls can break a brittle agent. Training against continuously changing generated interfaces could, in principle, force more general action understanding. That is speculative, but it is one of the more interesting secondary uses Runway calls out.

3. Design tooling already pushed visuals ahead of implementation. Image models make polished screens cheap. The bottleneck is turning those screens into working software. Interface World Models attack that bottleneck by refusing the translation step. Whether that is the right tradeoff depends on the domain—checkout flows and medical records are not the same as a brand experience or science demo.

Runway also ran a user study comparing Solaris responses to coded interfaces driven by a strong language model (Claude Opus 5 in their write-up), with 250 participants and thousands of pairwise judgments. Participants preferred Solaris for following the instruction and, more strongly, for natural in-scene behavior. Treat vendor studies carefully, but the gap they highlight is real: coded UIs often apply an isolated update; a world model can keep materials, lighting, and object relationships coherent because those priors are baked into generation.

What it cannot do yet (read this before planning a roadmap)

Runway is comparatively direct about limits, and those limits should drive any practical evaluation:

• Text. Stable, legible text is still one of the hardest problems in video generation, and interfaces depend on text. Hybrid approaches—image models for text-heavy pauses, video models for continuous motion—are one near-term path. Fully real-time generated text remains open.
• Trust and grounding. A convincing wrong answer is worse than no answer in instructional or commercial settings. Anchoring with real product imagery and verified context is necessary; richer grounding over a session is active research.
• Long sessions. Coherence over extended open-ended use is unfinished work. Short demos are easier than hour-long workflows.
• Accessibility and integration. A stream of pixels does not automatically expose roles, labels, and structure for screen readers and accessibility APIs. Generated interfaces still have to live inside the rest of the software stack.

Cost is also a standing concern. Generating every frame is inherently more expensive than serving a page built once. Runway says the same work that made Solaris real-time also made it far cheaper than a standard video diffusion path, and that the cost curve continues to improve. Teams should still model serving economics before assuming this replaces conventional front ends.

Image credit: Solaris scene example from Runway (source: runwayml.com)

Figure 3. Checkpoint list for evaluating Interface World Models as they leave research.

Practical implications for different readers

Product and design leaders: Treat Solaris as a signal, not a shipping dependency. The near-term wins look like immersive brand experiences, configurators, education demos, and marketing environments where “alive” visuals matter more than deterministic form submission. Keep high-stakes transactions on coded rails until text fidelity, auditability, and accessibility catch up.

Engineers: The architecture pattern to note is the LLM/world-model split. Even if you never use Runway, separating intent planning from perceptual rendering is a useful way to think about agentic UIs. Also watch how interaction is represented as conditioning rather than as a brittle DOM event map.

AI researchers and agent builders: The training-environment angle may matter as much as the consumer demo. Dynamic generated UIs could become a curriculum for computer-use agents that currently fail under slight layout shifts.

Security and trust teams: Open-ended generation expands the attack and failure surface. Prompt injection into the reasoning model, misleading visual grounding, and inaccessible critical controls are not edge cases—they are default risks for any interface that invents itself at runtime.

What to watch over the next few months

1. Public availability and API shape. Early access is partner-led today. Watch whether Runway ships a constrained product surface (for example, branded 3D-like shopping scenes) before a general “generate any app” claim.
2. Text and form handling. Until generated text is stable, expect hybrids that composite conventional text layers onto generated scenes.
3. Evaluation beyond preference studies. Instruction preference is useful; task completion, error rates, and accessibility conformance will decide enterprise adoption.
4. Competition from adjacent stacks. Real-time world models, on-device UI generators, and stronger multimodal coding agents are all aiming at pieces of the same problem from different directions.
5. Standards for provenance. If interfaces can be synthesized on the fly, users and regulators will ask what was verified, what was invented, and how to reproduce a session for support or compliance.

Bottom line

Solaris is a clear statement of direction from a company that already ships generative video: push world models into the operating layer of software so interfaces are generated as they are used. The useful mental model is not “AI redesigns my Figma file.” It is “the front end becomes a real-time generative process steered by an LLM and conditioned on user actions.”

That idea is timely because design fidelity from generative models has outpaced the reliability of turning those designs into software, and because agents still struggle with brittle UIs. It is also incomplete: text, trust, long-horizon coherence, accessibility, and cost remain first-order blockers for everyday applications.

For now, the right move for most teams is to study the architecture, prototype only in domains where visual flexibility outweighs deterministic guarantees, and track whether Interface World Models graduate from partner demos into systems that can be tested, audited, and operated like real software. The announcement is new as of August 31, 2026. The questions it raises—about what an “app” even is—will stick around longer than the launch cycle.

Source: Runway, “Introducing Solaris,” August 31, 2026 — runwayml.com/news/research/introducing-solaris