EVOKE: Eliciting World Knowledge in Agents for Transferable Decision-Making

0:00Welcome to Daily Papercast. Today's paper is from the Hugging Face Daily Paper List of October 1st, 2026, and has received 37 upvotes. The title of the paper is, evoke, eliciting world knowledge in agents for transferable decision making. It's authored by Yu Han Guol and Jin Ming Liu, with corresponding author Wen Jun Zang from Shanghai Jiao Tong University, Hong Kong Polytechnic University, and Eastern Institute of Technology, Ning Bo. Let's dive into the introduction of this paper. Large language models, or LLMs, have been increasingly trained to handle multi-step decision making in interactive environments. Why is this important, Ashley? Well, Evan, it's crucial because these agents often perform quite well in the environments they've been trained in. However, their performance significantly dwindles in unseen environments. That makes sense. This challenge arises because their learning is closely tied to specific tasks and environments encountered during training, right?

1:07Exactly. One proposed solution to enhance transferability is enabling the agent to predict the consequences of actions before choosing them. This approach leverages rules that generally hold true across various environments, increasing the likelihood that decisions can be generalized. So, the idea is that understanding action consequences allows for better decision making. What methods have been traditionally used to achieve this? Traditionally, world model methods have been utilized, which train agents to predict future observations based on their actions. However, this approach has two main drawbacks. Oh, what are those? First, it adds an extra prediction objective which makes training more expensive. Second, errors from predicted futures tend to compound over successive steps, leading to inaccurate planning. That sounds quite inefficient. So, do LLMs necessarily need these predictive abilities to understand action consequences?

2:08Interestingly, they might not. For agents operating in digital environments, much of the world knowledge needed for understanding these consequences is already internalized during pre-training. I see. Instead of acquiring new knowledge, the focus shifts to eliciting pre-existing knowledge within the models. Yes, the paper argues that typical post-training offers little pressure for elicitation. As supervision under a single goal, drives policies to rely on superficial contextual habits rather than utilizing deeper world knowledge. So, what approach does the paper propose to address this issue? The paper introduces a method called evoke. It's a post-training technique that increases this pressure through goal diversity at fixed states. How exactly does evoke work? Evoke holds the environment state an interaction history fixed and ranks the same candidate actions under alternative goals. Because the action preferences change depending on the goal, a policy relying on contextual habits won't be able to order them correctly.

3:12This helps elicit the pre-trained world knowledge to inform decisions. That's a clever approach. What kind of evaluation did the authors use to test this method? They evaluated evoke across various tasks using three different backbone models. They found that the method improves task performance, generalization to unseen environments, and data efficiency. And what contributions do the authors highlight from their work? They emphasize three main contributions. Revisiting how LLM agents can benefit from world knowledge, introducing evoke to leverage goal diversity for decision level supervision and demonstrating the effectiveness of the method on diverse tasks. That's the end of the introduction section. Let's delve into the method proposed in the paper called evoke. Ashley, can you explain the foundational components of this approach? Sure, Evan. The policy input for evoke consists of a goal, the visible interaction history,

4:12and a list of available actions. The output is the next action as text. Initially, the policy is obtained by supervised fine-tuning on successful demonstrations. That sounds like a standard starting point. What happens next? Next, decision state collection takes place. The policy is run in training environments, and at each step, the goal and the visible history are recorded. This ensures that the state can be revisited for further training. And how are these decision states chosen? States are sampled along trajectories to cover both progress-making steps, and those that precede detours, repeated actions, or failure. This way, supervision addresses situations the policy actually encounters. That makes sense. So once decision states are collected, what's the next step? Following that, goal interventions are performed. For each collected state, the original goal is retained, and alternative goals achievable from the same state are added. An annotator, typically a large language model,

5:15proposes candidates for these alternative goals. What criteria do these alternative goals need to meet? Great question. The alternative goals must be compatible with the scene, not yet satisfied, and achievable from the current state. The original task text is removed from the visible history to prevent the policy from relying on it. And how are the candidate actions evaluated in this setup? Candidate actions are evaluated by executing each from the same state and observing the resulting outcomes. The annotator then labels actions that advance the goal as positives and others as competitors. Actions with insufficient evidence are discarded. So the annotator judges the outcomes based on actual execution rather than predictions. That seems robust. What about ensuring the policy learns effectively from these annotations? Indeed, the policy undergoes contrastive ranking where actions are scored and ranked based on mean-token log probabilities. A list-wise and pair-wise loss is used to ensure that several actions can be correct,

6:20but separates each positive from each competitor. How are these scores and rankings used to update the policy? The scores and rankings are used to update Laura Adapter's attached to the model. This is done iteratively, with new states and mistakes being reached in each round, which then become new hard negatives for further training. It seems iterative improvement is a key part of this method. How many rounds are typically conducted? The paper generally conducts multiple rounds. Each round aggregates data from previous rounds, ensuring the policy is continuously refined, and hard negatives are aligned with current policy errors. Does the policy require any external modules or inference-time planning after training? No, it doesn't. At deployment, the policy operates from the goal, history, and available actions alone, without requiring any world model module or inference-time planning. That's quite efficient.

7:20What about the evaluation setup? How did they test evoke? Evoke was evaluated across several tasks using three different backbone models. The tasks included household activities in Alphworld, search-based Q&A, and web navigation using Webshop. What metrics were used to gauge performance? The paper used task success rates, data efficiency, and generalization to unseen environments as primary metrics. For example, Alphworld evaluates success across tasks like pick, look, clean, and heat with scene and unseen games reported separately. And did evoke show improvements across these metrics? Yes, the results showed that evoke-improved task performance generalized better to unseen environments, and was more data efficient compared to several baseline methods. Can you specify what baseline's evoke was compared against? The paper-compared evoke with prompting methods such as vanilla and react,

8:21policy optimization, and distillation methods like GRPO and RLSD, and world model methods including IWM and MEMWM. How did the performance of evoke fare against these diverse baselines? Evoke achieved the best or second best results across all benchmarks on each backbone model, demonstrating its robustness in a variety of tasks. It consistently outperformed other methods in terms of task completion rates and generalization to unseen scenarios. Impressive. And what can we conclude about the advantages of goal diversity? The introduction of goal diversity at fixed states helps the policy utilize its pre-trained world knowledge more effectively, leading to better performance and generalization. It pressures the policy to consider the implications of actions more deeply, rather than relying on contextual habits. That's the end of the method section. Let's move on to the experiments and results section.

9:21Ashley, how did the authors assess the performance of evoke? The authors evaluated the performance of evoke across multiple benchmarks using three different backbone models, QN2.5.3B, QN2.5.7B, and QN3.1.7B. They focused on tasks such as household activities within Alphworld, web navigation with webshop, and search-based Q&A. What metrics did they use to measure success in these experiments? The primary metrics were task success rates, generalization to unseen environments, and data efficiency. These metrics provided a comprehensive view of how well evoke could handle diverse, real-world tasks. Could you provide some details on how evoke performed in each benchmark? Let's break it down by benchmark. First, we have ALFworld, which consists of tasks like pick, look, clean, heat, cool, and pick two. Here, evoke demonstrated substantial improvements in both

10:24scene and unseen games. For example, in the unseen task setting, evoke achieved success rates as high as 95.8% in tasks like pick and cool. That's quite remarkable. How did evoke fair in the webshop benchmark? In the webshop benchmark, which involves interactions like searching for items and making purchases, evoke again showed strong performance. It achieved a mean task score of 90.6% and a task success rate of 85.9% on the QN2.5.7B backbone, outperforming all other methods under comparison. And what about the search-based Q&A tasks? For search-based Q&A tasks, evoke led in benchmarks such as natural questions, trivia QA, and hotpot QA. It achieved consistently high answer accuracy across these data sets, notably scoring 49.2% in the aggregated metric on the QN2.5.7B backbone.

11:27That's impressive across various types of tasks. How did evoke compare to other methods in terms of data efficiency? Evoke showed significant advantages in data efficiency. For instance, even with 60% of the labeled states used in training, evoke outperformed imitation-based methods trained on all available data. This indicates that evoke can leverage data more effectively, extracting more learning from each labeled state. That's quite efficient. Did the authors conduct any specific analyses to identify the sources of these performance gains? Yes, they did. The authors ran controlled analyses to pinpoint the factors driving evokes improvements. They found that the introduction of alternative goals at fixed states was critical. In comparisons where only original goals were used, performance dropped significantly. It seems goal diversity played a crucial role. Were there any other noteworthy findings from these analyses? Indeed. Another key

12:29finding was the impact of iterative preference learning. Each round of training captured new states and mistakes, which were valuable for further refining the policy. The continuous aggregation of data across rounds also ensured hard negatives were aligned with the current policies errors. So, iterative rounds are beneficial for capturing an evolving set of mistakes and correcting them? Precisely. This iterative approach led to consistent improvements across all backbone models, showcasing its robustness in adapting to errors and new scenarios. Did the authors examine if the pre-trained models already encoded the needed world knowledge? Yes, they did. Diagnostic analyses showed that the pre-trained backbone models could almost perfectly predict action consequences before any task-specific training. This suggests that evoke success is due to its ability to elicit and apply this pre-existing knowledge during decision making. That's a fascinating insight. So, the pre-trained models have the knowledge and evoke

13:36effectively harnesses it? Exactly. The policy learns to make goal-directed decisions rather than following contextual habits. Tests involving alternative goal pairs showed that evoke minimized habitual errors significantly compared to other methods. Moreover, how did evoke perform in practical execution scenarios? Evoke not only succeeded in more tasks, but also did so more efficiently. It completed tasks faster, made fewer invalid actions, and revisited locations less frequently. This demonstrates that evoke led to more strategic and effective decision paths. That's the end of the experiment section. Now, let's discuss the related work section of the paper. Ashley, can you give us an overview of the foundational research that evoke builds upon? Evan, the paper reviews several key areas of research that are foundational to evoke's development. These include world-model methods for

14:36LLM agents, policy optimization techniques, and methods for goal-conditioned learning. Let's start with world models. What are world models? And how have they been used in the context of LLM agents? World models equip agents with knowledge about the consequences of their actions. Some methods build explicit models for lookahead, combining pre-trained knowledge with symbolic rules. For example, Wally integrates pre-trained knowledge with symbolic dynamics, while ITP learns textual dynamics for adaptive lookahead. So these models help agents predict what will happen next. Are there other methods within this domain? Yes, others use auxiliary prediction signals to enhance policy learning. For instance, IWM learns next state prediction before imitation learning, and PODOW optimizes both observation prediction and policy learning. Additionally, methods like NVRL add forward and inverse dynamics to improve task performance.

15:38It seems like a lot of effort goes into predicting future observations. How does evoke differ from these approaches? Evoke takes a different route by leveraging the world knowledge already present in pre-trained models, instead of acquiring new knowledge through predictions. It elicits this existing knowledge through goal diversity and decision level supervision, avoiding the compounding errors and extra training costs associated with prediction-based methods. That's a clever approach. Moving on, let's talk about policy optimization for LLM agents. What are some methods used to enhance agent policies? Policy optimization methods improve agents by learning from their own interactions. Reinforcement learning, for example, optimizes policies using task rewards, employing group relative estimates at the trajectory or step level. Distillation methods add guidance from teacher signals refining the policy further. Can you mention a few specific techniques within policy optimization?

16:38Sure. ETO learns from preferences between successful and failed trajectories, while Dagger aggregates labels at learner-visited states. These methods differ mainly in how they compute the training signal. Evoke focuses on a complementary aspect, the goals under which each visited state is supervised, adding pressure through goal diversity. Interesting. Now, what about goal-condition learning? How does it integrate with evokes methodology? Goal-conditioned reinforcement learning shares experience across different goals. Techniques like universal value functions generalize over goals, and successor features separate environment dynamics from task-specific rewards to enable transfer. Hindsight experience replay relabels trajectories with achieved goals, allowing even failures to provide learning signals. Can you name some methods within goal-condition learning? Certainly. Hindsight supervise learning relabels language agent trajectories by constructing demonstrations for achieved goals. Richens at all have shown that an agent competent

17:43across a rich set of goals must contain a world model recoverable from its goal-conditioned behavior. Evoke applies goal diversity at a single decision, assessing actions under alternative goals for robust preference learning. It sounds like evoke leverages several insights from these related works, but also brings something new to the table. How does evoke combine these elements effectively? Evoke combines these elements by focusing on eliciting existing world knowledge through a targeted method of goal diversity at fixed states. This approach forces the policy to engage with the depth of its pre-trained knowledge, rather than memorizing surface-level behaviors, bridging insights from world model prediction, policy optimization, and goal-condition learning. That's the end of the related work section. Let's summarize the key contributions and takeaways of the evoke paper. Ashley, what stands out to you in this research? The paper introduces a novel approach to enhancing the decision-making capabilities of large

18:48language models by eliciting the world knowledge already present in pre-trained models. Instead of relying on prediction-based methods to learn action consequences, evoke leverages goal diversity at fixed states to uncover this knowledge. That's a significant shift. What are the main benefits of this method? Evoke demonstrates notable improvements across several key areas. Firstly, it improves task performance in diverse environments, both scene and unseen. Secondly, it significantly enhances generalization, allowing models to transfer their learning effectively to new contexts. Lastly, it boosts state efficiency, extracting more value from each labeled state. It sounds like evoke provides a robust solution to some of the challenges faced by LLM agents. Any final thoughts on the methods impact? The methods ability to make goal-directed decisions rather than relying on contextual habits is a major leap in the field. By iteratively refining the policy through exposure to

19:49alternative goals and hard negatives, evoke ensures continuous improvement. That's a wrap on today's discussion. We've covered the innovative evoke method for eliciting world knowledge in LLM agents, which holds great promise for enhancing their decision-making capabilities. We hope you found our deep dive into this paper both informative and engaging. Remember, we'll be back with more insights into cutting-edge research from HuggingFaces Daily Paper List. Thank you for listening to Daily Papercast. Be sure to join us in our next episode for more on the latest advancements in AI and machine learning. Until then, stay curious and keep exploring the world of AI. Goodbye for now. Goodbye.