[GSoC] Trigger-based evaluation in behavioral frameworks

[souro26]

Builds on #2538 #3304 #2529

Related discussions #3443

While reading through the recent discussions around behavioral frameworks for my GSoC proposal, I started looking into a design question around when agent decision logic should actually be evaluated. In many examples, decision logic is evaluated on a recurring schedule. For example:

model.schedule_recurring(agent.decision_system.evaluate)

This works well and is straightforward, but it also means the evaluation runs every step, even when nothing relevant has changed in the agent’s state or environment. To explore this a bit further, I ran a small experiment using the WolfSheep model to measure how frequently decision logic actually gets evaluated under different approaches.

This experiment is only meant to observe evaluation frequency, not runtime performance.

My Experiment

The goal of the experiment was simply to count how often decision logic gets evaluated.

1. Polling evaluation (baseline)

The baseline evaluates decision logic every step.
Example run:

Total decision evaluations: 3720
Total agents: 400
Steps: 200

2. Trigger-driven evaluation

In the second version, evaluation is requested only when certain state conditions occur..
Example run:

Total decision evaluations: 168
Total agents: 538
Steps: 200

This configuration intentionally contains very few triggers, so evaluation requests happen rarely.

3. Trigger-driven evaluation with periodic checks

To avoid agents remaining inactive for long periods, I added a small probabilistic reevaluation trigger:

self.random.random() < 0.05

Example run:

Total decision evaluations: 1139
Total agents: 479
Steps: 200

Observation

Across these runs, the number of evaluations varies depending on the trigger policy. Even with simple trigger mechanisms, evaluation frequency can be significantly lower than evaluating every step.

Concept

Based on this experiment, one possible structure is a trigger-driven evaluation layer for behavioral frameworks. Instead of evaluating decision logic every step, evaluation could be requested by events or state changes. Possible triggers include:

• state thresholds
• task completion
• environment events
• periodic checks

These triggers request evaluation through a scheduler that determines when the agent’s decision logic should run. Importantly, this layer is independent of how the decision logic itself is implemented. The decision system could still be rule-based, RL-based, or implemented using other behavioral frameworks.

Prototype sketch

To explore the idea in practice, I implemented a small prototype on top of the Wolf–Sheep model. Instead of evaluating decision logic every step for every agent, agents request evaluation when certain conditions occur for example energy thresholds or periodic checks. A simple scheduler then executes evaluation only for the agents that triggered a request.

A simplified version of the prototype logic looks like this

for sheep in self.agents_by_type[Sheep]:
    if sheep.energy < 5 or self.random.random() < 0.05:
        self.behavior_scheduler.request(sheep)

for wolf in self.agents_by_type[Wolf]:
    if wolf.energy < 10 or self.random.random() < 0.05:
        self.behavior_scheduler.request(wolf)

self.behavior_scheduler.run()

In this prototype agents request evaluation when a trigger condition occurs, the scheduler collects these requests andevaluation is executed only for the requested agents.

Conceptual architecture

I sketched a small diagram showing the idea.

In this structure triggers request evaluation, an evaluation scheduler decides when evaluation runs, the scheduler invokes the agent’s decision logic and the decision logic produces actions which update the agent state. The evaluation layer itself remains generic and could support different decision mechanisms such as:

• rule-based systems
• reinforcement learning policies
• behavior trees
• other behavioral frameworks

Current state

Right now this is only a small prototype used to explore the idea and measure evaluation frequency. A full implementation would need to address several questions, including:

• how triggers are registered and managed
• how evaluations interact with model time
• how this integrates with Mesa’s event system
• how to prevent excessive trigger generation

I still have a lot of work left, but I would appreciate any feedback on this idea. Any thoughts or suggestions would be extremely helpful. Thanks.

[souro26]

I spent some more time playing around with the trigger-based evaluation idea and was thinking about how the architecture could actually look like. In many models, decision logic is evaluated on a recurring schedule, something like,

model.schedule_recurring(agent.decision_system.evaluate)

This works well, but it also means evaluation is always driven by time steps, even when nothing relevant about the agent or the environment has changed. A lot of simulation behaviors are naturally state driven, not time driven. For example, an agent becoming hungry, a population crossing a threshold, a resource becoming available or an agent entering or leaving a certain state. These situations are usually written currently as recurring checks.

def check_condition():
    if agent.energy < 5:
        agent.evaluate_behavior()

model.schedule_recurring(check_condition)

Conceptually though, the logic is closer to running an evaluation when this condition becomes true.

Prototype exploration

I built a small prototype layer with three simple components: StateTrigger, TriggerRegistry and BehaviorScheduler. The basic flow looks like this

flowchart TD
    A[State change] --> B[StateTrigger detects condition]
    B --> C[Evaluation request queued]
    C --> D[BehaviorScheduler executes evaluation]

Loading

Example concept for this is,

trigger = StateTrigger(
    condition=lambda agent: agent.energy < 5,
    action=agent.evaluate_behavior
)

So instead of evaluating decision logic every step, evaluation is requested by triggers when something relevant happens.

Example usage

A minimal example of how these components could be used together might look something like this

scheduler = BehaviorScheduler()
registry = TriggerRegistry()

trigger = StateTrigger(
    condition=lambda agent: agent.energy < 5,
    action=lambda agent: scheduler.request(agent.evaluate_behavior)
)

registry.register(trigger)


registry.notify_state_change(agent)
scheduler.run()

Observations

The current prototype still checks triggers once per step, which technically means it is still polling internally. That raised a more interesting design question. Rather than checking triggers every step, should triggers integrate with the event system? Conceptually this would look like

flowchart TD
    A[State mutation] --> B[StateTrigger]
    B --> C[Event scheduled]
    C --> D[mesa.time event queue]
    D --> E[Evaluation executed]

Loading

So evaluation remains event driven and uses the existing event queue.

Trigger behavior

Another thing that came up is how triggers should behave once the condition becomes true. Possible directions include

• fire once when the condition first becomes true
• fire every step while the condition remains true
• fire with some cooldown

A useful default might be something like edge triggered behavior. The trigger fires when the condition changes from false to true, and does not fire again while the condition remains true. This avoids repeatedly triggering the same evaluation.

Separation of concerns

Triggers can handle when evaluation happens, while the decision system itself handles what action to take. Conceptually, the decision system decides what to do, while the trigger system decides when evaluation happens. That separation could allow different behavioral systems to plug into the same scheduling mechanism.

@quaquel and @EwoutH i would really appreciate any thoughts and suggestions you might have in this direction, thanks.