AgentGarten: Code Worlds for Evolving Agents

0:00Welcome to Daily Papercast. Today's paper is from the Hugging Face Daily Paper List of October 9, 2026, and has garnered 96 upvotes. The title of the paper is Agent Garden, Code Worlds for Evolving Agents. Authored by Jiau Hui Chi and Shang Chen Mi Yao, with corresponding author Fang Fu Liu from Miros Lab. Let's dive right into the introduction. Ashley, what's this paper about? This paper is centered around creating interactive virtual worlds that facilitate agent learning through exploration and interaction. Okay, and what limitations do current environments have? Traditional environments often struggle to offer both consistency and realistic visual observations. These are crucial for agents to learn effectively. Interesting. So, what's the proposed solution in this paper?

1:00They introduce Agent Garden, a framework that combines simulators and game engines with a shared neural renderer to build real-time interactive environments. How does Agent Garden achieve this integration? The simulation back ends in Agent Garden, maintain a persistent world state, and execute rules for interaction. While the renderer generates visual observations from structured conditions exported through a common interface. And how do they make the neural renderer work with various geometries? They adapt a pre-trained video model to geometry conditions and use a method called adversarial forcing. This technique makes history prefilling differentiable through exact replay and boosts visual quality via adversarial supervision. So, how does this approach improve the learning process of agents? Agents in Agent Garden perceive the world through these generated visual observations, interact with it in real-time, and improve by distilling each round of experience into playbooks, subsequent agents then inherit and refine these playbooks.

2:07Sounds efficient. What kind of efficiency improvements does it offer? Their empirical study shows that agents can learn from just four rounds compared to millions of rounds required by conventional reinforcement learning methods. That's quite a leap. Can new worlds be added to this system easily? New worlds can be coded and rendered through the same interface, allowing environments to scale both in number and difficulty along with the agents. It seems like Agent Garden could be a game changer for AI learning environments. Any concluding thoughts on this introduction? The introduction of Agent Garden signifies a step toward more responsive and evolving agents through interactive experiences. And that's the end of the introduction section. Alright, Ashley, let's move on to the method section of the paper. How does Agent Garden pull everything together? Agent Garden operates through what the paper calls code worlds, which merge scene programs, simulation engines, and a neural renderer.

3:09Code worlds, you say? How are these structured? A code world is made up of a scene program, an engine that executes it, and a neural renderer called R Theta. The scene program defines the layout and interaction rules of the world. So, how does the engine interact with the scene program? The engine keeps track of the scene state, which includes object positions and any articulation or task-specific variables. It also handles camera perspectives and the agents actions. And these actions and states, how are they managed or updated? The state and actions are updated through predefined conditions, which are then fed into the neural renderer. The renderer then generates the visual observations based on these conditions. Interesting. So, the renderer plays a central role here. How does it maintain consistency and quality of these visual observations? They use a method called adversarial forcing to train the renderer.

4:12This technique involves exact replay, where losses on later predictions update the history, and an adversarial objective to maintain high visual fidelity. Could you break down adversarial forcing for us a bit more? Sure. Adversarial forcing trains the neural renderer using its own generated rollouts. It updates the history without high memory costs, and employs a real data adversarial objective to enhance visual quality. That sounds robust. What specific training stages does the renderer go through? The training is divided into three stages. First is geometry conditioning, where the renderer learns to use depth or surface normals as structured conditions. The second stage is teacher forcing adaptation, where the model adapts to generating sequences blockwise. Teacher forcing, you say? What does that involve? It involves using two copies of each block inside one transformer, a claim copy representing history, and a noisy copy that is denoised.

5:14So this method lets all target blocks be trained in parallel. OK. And what happens in the third stage? The third stage is adversarial forcing, where the model is distilled into a real time renderer on its own rollouts. This stage involves distribution matching, exact replay for history gradients, and using a real data adversarial objective with R1, R2 regularization. And what does this exact replay accomplish? Exact replay ensures that gradients propagate through the entire history encoding computation. This is done without the high memory cost associated with full differential rollouts, keeping the computations accurate. How do these training stages impact the performance? The end result is a high fidelity neural renderer that operates in real time, providing immediate feedback to agents essential for their learning processes. This setup sounds really comprehensive. What about the agents themselves? How do they learn and adapt?

6:17Agents learn through a continuous dynamic process. They act on the environment, perceive the changes through visual observations generated by the renderer, and refine their strategies based on these interactions. You mentioned something about playbooks earlier. Can you elaborate on that? Yes, after each round, agents compile their experiences into written playbooks, which include their observations, failed attempts, and refined strategies. These playbooks are then passed on to subsequent agents who inherit and improve upon them. So essentially, these playbooks serve as a repository of learned skills and strategies. Exactly, and this iterative process shows significant efficiency improvements, with agents learning complex tasks in just a few rounds compared to the millions required by traditional methods. How are these environments set up initially? Do they require extensive manual coding? Not necessarily. Agent Garten supports automated environment generation.

7:19Coding agents can build a scene program from a single image or a text description, effectively leveraging perception and generation models as tools. And how do they ensure the scene is accurate and functional? The agent inspects rendered views and performs rollout checks. It then repairs any identified issues in geometry, supports, or occlusion to ensure the environment is accurate and functional. This seems like it would be quite adaptable. Can these environments be extended or modified easily? Yes, the environments are coded, so modifications and extensions can be made easily. The same coding methods can introduce new elements to existing worlds, enhancing their complexity or introducing new challenges. That sounds incredibly flexible. So to wrap up, how does this method contribute to evolving agents overall? By providing a scalable and adaptable platform with a high fidelity renderer, Agent Garten allows agents to continuously evolve.

8:22They interact with well-defined, accurate environments, learn from each experience, and pass their insights forward. Indeed, it seems like a promising step forward for AI learning, and that concludes the method section. Alright, Ashley, let's delve into the experiment and result section of the paper. What kind of experiments were conducted to evaluate Agent Garten? To evaluate Agent Garten effectively, the authors designed a variety of experiments across different environments, focusing on visual quality, replay accuracy, and inference throughput. What specific aspects of visual quality did they examine? The visual quality was assessed by comparing long rollouts from renderers trained with different methods, specifically self-forcing and adversarial forcing. The results showed that adversarial forcing managed to retain natural textures and fine details throughout an extended interaction, unlike self-forcing, which developed repetitive surface patterns over time.

9:25Retaining visual quality over long durations sounds crucial for realistic agent interactions. What about the accuracy of replaying these states? Replay accuracy was tested by measuring the relative L2 error between rollout and replay. The paper mentions that while the flux attention replay method showed a 3.99% relative error, the block-by-block-scaled dot-product attention, or SDPA, replay method achieved Bitwise Identical Accuracy, which means there was zero error and better computational efficiency. A Bitwise Identical Replay is impressive. And how did they handle throughput concerns? For inference throughput, the paper measures the steady state performance and breaks down the inference cost using 1 Nvidia H100 GPU. The renderer, with the help of handwritten kernels and cuda graph capture, achieved over 35 frames per second at a resolution of 480 by 832 pixels, ensuring prompt response for agent actions.

10:28This sounds very efficient. Now, were any specific environments chosen for these experiments? Yes, the authors created and tested several diverse environments – navigation, manipulation, tool use, multi-agent interaction, and vehicle maneuvering. They even ran browser games like bowling and a crate vault puzzle to demonstrate interactive deployment through WebRTC streaming. That's a diverse set of environments. How does agent-garten handle complex behaviors like strategy emergence? One notable experiment revisited OpenAI's hide and seek scenario with a sequential one-hider, one-seeker variant. This set up allowed agents to develop strategies like building shelters and using ramps to overcome obstacles. Remarkably, these strategies emerged within just four rounds for hideers and 10 rounds for seekers, showcasing significant improvement compared to the 25 million and 100 million episodes reported by OpenAI's previous self-play reinforcement learning.

11:30That's quite a reduction in required episodes. How do the playbooks fit into this process? After each round, agents write and refine playbooks that document the strategies they tried, their failures, and observations. These playbooks provide a repository of skills that subsequent agents inherit and further refine. So the playbooks facilitate ongoing learning and improvement. What other worlds were explored using this procedure? Beyond hide and seek, four additional worlds were tested, companion dog engagement, a cooperative one-lane bridge for car navigation, hurting sheep into a pen, and operating a quarry loader to move rocks and park correctly. Each world included its own set of actions and goals, which tested the flexibility and adaptive capabilities of the agents. And what were the results across these diverse scenarios? Each world showed measurable improvement over rounds. For example, in the companion dog world, engagement scores rose from 13 to 19 across rounds.

12:32In the one-lane bridge scenario, the completion time decreased from 71 to 41 seconds. Herding demonstrated improved control, with all sheep being penned successfully from the second round onward. And in the quarry loader experiment, the agents progressively achieved more complex tasks within their time limits. These results highlight substantial improvements in agent performance within a short time span. Any final notes on the empirical findings? The experiments demonstrate that agent gardens combination of code worlds and neural rendering significantly boosts agents' learning efficiency and adapts them to increasingly complex scenarios. By leveraging playbooks in real-time visual interactions, agents show enhanced development and response to their environments. Indeed, it's fascinating to see such advancements, and that wraps up the experiment section. Now that we've covered the experimental setup and results, Ashley, can you take us through the related work section of the paper?

13:33Of course. The related work section in this paper is quite comprehensive, and it provides a solid backdrop for understanding the unique contributions of agent garden. Great. Let's dive in. What are some of the foundational works in this domain? The paper starts by discussing executable environments and code worlds, which have been essential for embodied learning. For example, Habitat 2.0 enables household rearrangement by simulated robots, and Monyskilt 2 offers benchmarks for diverse manipulation tasks. Interesting. These environments essentially create tailored simulations for specific tasks. Exactly. Another relevant work is Prokthor, which generates interactive houses procedurally at scale. Additionally, frameworks like SceneCraft and LLMR use language models to synthesize and revise environment scenes automatically. So, these systems automate the creation and customization of environments. How does agent garden differentiate itself from these approaches?

14:37What sets agent garden apart is its combination of real-time neural rendering with code worlds, which provides a scalable and adaptable platform. It emphasizes visual fidelity and historical consistency, which are often limitations in other models. You mentioned neural rendering. How does the paper relate agent garden to previous work in this area? The paper references recent work that keeps the state of the world outside the video model and uses the model to render it. Some systems learn the state jointly with the frames, while others advance the state through an engine or agent-written code. And what specific methodologies do they discuss in relation to neural and generative rendering? They compare it to projects like Genie, which learns latent actions from unlabeled video and Game Engine, which simulates doom from action observation trajectories, both aim to create interactive environments from video data. It seems that there's a significant focus on rendering from course or intermediate representations. How does agent garden leverage this concept?

15:48Agent garden uses structured conditions, such as depth and surface normals, which can be estimated from video or rendered by an engine. This allows for real-time generation of visually consistent observations, a point where many models fall short. Maintaining visual consistency is indeed crucial. Are there any notable advancements in training methodologies that are highlighted? Yes, the paper delves into autoregressive video generation and distillation methods. One pertinent technique is self-forcing, which trains a model on its own rollouts rather than on ground truth prefixes. This method is further enhanced with adversarial forcing, which matches distributions and replays history accurately. That sounds quite advanced. Are there any specific neural network architectures or optimization strategies mentioned? Indeed, they adapt techniques like Cosvid, which turns a bidirectional video model into a causal student.

16:49The training stage is involved geometry conditioning, teacher forcing adaptation, and adversarial forcing to achieve high fidelity and real-time performance. It's quite clear that agent garden builds upon a rich foundation of previous research. Any closing thoughts from the related work section? The related work section highlights that agent garden leverages and advances multiple areas, including executable environments, neural rendering, and training methodologies to provide a new framework for evolving agents. Thanks, Ashley. That concludes the related work section. Now, let's move on to the conclusion of the paper. Alright, Ashley, we've gone through a lot of details. Let's summarize the key contributions and takeaways of the paper, agent garden, code worlds for evolving agents. Sure, Evan. To start, the paper introduces agent garden, which merges simulators and game engines with a shared neural renderer to create real-time interactive environments.

17:51This setup allows agents to learn efficiently through exploration and interaction. Exactly. By using a method called adversarial forcing, agent garden ensures high-quality visual observations and consistent world states, addressing the limitations of traditional environments. Right. Another significant contribution is the concept of code worlds. These are dynamic environments defined by scene programs that handle layouts and interaction rules, which makes them easy to modify and extend. And importantly, the agents continuously evolve by documenting their experiences in playbooks. These playbooks are passed on to future agents, drastically improving their learning efficiency. The experiments conducted support this, showing noticeable improvements in agent performance across various tasks, from hide and seek to complex tool use and manipulation tasks. In just a few rounds, agents were able to develop sophisticated strategies that would typically require millions of episodes in traditional reinforcement learning setups.

18:58Overall, agent garden proposes a scalable, adaptable, and efficient method to evolve agents more rapidly and effectively through rich, interactive experiences. That's a wrap for today's episode. We hope you found our discussion on agent garden insightful. Be sure to tune in to our next episode for more on the latest advancements in AI and machine learning research. Thanks for listening to Daily Papercast and have a great day. See you next time.