Human-in-the-Loop Tool Confirmation

Edit this page

When AI agents execute tools, some actions — like deleting files, sending emails, or modifying data — should require human approval. This guide shows how to build a confirmation system using the ToolCallRequested event.

Prerequisites

  • Symfony AI Platform component
  • Symfony AI Agent component
  • Symfony EventDispatcher component

Step 1: Install Packages

Install the Platform and Agent components together with the EventDispatcher:

1
composer require symfony/ai-platform symfony/ai-agent symfony/event-dispatcher

Step 2: Confirm Every Tool Call

The Toolbox dispatches a ToolCallRequested event before each tool execution. An event listener can inspect the tool call and either:

  • Allow it — do nothing, the tool executes normally
  • Deny it — call $event->deny($reason) to block execution and return the reason to the LLM
  • Replace it — call $event->setResult($result) to skip execution and return a custom result

The simplest approach asks for confirmation on every tool call:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
use Symfony\AI\Agent\Toolbox\Event\ToolCallRequested;
use Symfony\Component\EventDispatcher\EventDispatcher;

$dispatcher = new EventDispatcher();
$dispatcher->addListener(ToolCallRequested::class, function (ToolCallRequested $event): void {
    $toolCall = $event->getToolCall();

    echo \sprintf(
        "Tool '%s' wants to execute with args: %s\nAllow? [y/N] ",
        $toolCall->getName(),
        json_encode($toolCall->getArguments())
    );

    $input = strtolower(trim(fgets(\STDIN)));

    if ('y' !== $input) {
        $event->deny('User denied tool execution.');
    }
});

Pass this dispatcher to the Toolbox:

1
2
3
use Symfony\AI\Agent\Toolbox\Toolbox;

$toolbox = new Toolbox($tools, eventDispatcher: $dispatcher);

Step 3: Add a Policy

In practice, you don't want to confirm every single call. A policy decides which tools need confirmation and which can run automatically:

1
2
3
4
5
6
7
8
9
10
11
enum PolicyDecision
{
    case Allow;
    case Deny;
    case AskUser;
}

interface PolicyInterface
{
    public function decide(ToolCall $toolCall): PolicyDecision;
}

A simple policy could auto-allow read operations based on tool name patterns:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
use Symfony\AI\Platform\Result\ToolCall;

class ReadAllowPolicy implements PolicyInterface
{
    public function decide(ToolCall $toolCall): PolicyDecision
    {
        $name = strtolower($toolCall->getName());

        foreach (['read', 'get', 'list', 'search', 'find', 'show'] as $pattern) {
            if (str_contains($name, $pattern)) {
                return PolicyDecision::Allow;
            }
        }

        return PolicyDecision::AskUser;
    }
}

Step 4: Build a Confirmation Handler

The confirmation handler prompts the user and returns a decision. Its implementation depends on your application context — CLI, web, async, etc.:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
use Symfony\AI\Platform\Result\ToolCall;

class CliConfirmationHandler
{
    public function confirm(ToolCall $toolCall): bool
    {
        echo \sprintf(
            "Allow tool '%s' with args %s? [y/N] ",
            $toolCall->getName(),
            json_encode($toolCall->getArguments())
        );

        return 'y' === strtolower(trim(fgets(\STDIN)));
    }
}

For web applications, you might store pending confirmations in a database and wait for a user response through an HTTP endpoint or WebSocket.

Tip

The event also exposes the tool's definition via $event->getDefinition(), which returns a Tool object whose getDescription() and getParameters() methods expose the tool's description and parameter schema. Use it to show the user more context before they decide.

Step 5: Wire the Policy and Handler Together

Combine the policy and handler in an event listener:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
use Symfony\AI\Agent\Toolbox\Event\ToolCallRequested;

$policy = new ReadAllowPolicy();
$handler = new CliConfirmationHandler();

$dispatcher->addListener(ToolCallRequested::class, function (ToolCallRequested $event) use ($policy, $handler): void {
    $decision = $policy->decide($event->getToolCall());

    if (PolicyDecision::Allow === $decision) {
        return; // Auto-approved, proceed with execution
    }

    if (PolicyDecision::Deny === $decision) {
        $event->deny('Tool blocked by policy.');

        return;
    }

    // PolicyDecision::AskUser
    if (!$handler->confirm($event->getToolCall())) {
        $event->deny('User denied tool execution.');
    }
});

Step 6: Remember User Decisions

To avoid asking the user repeatedly for the same tool, cache decisions:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
use Symfony\AI\Agent\Toolbox\Event\ToolCallRequested;

$decisions = [];

$dispatcher->addListener(ToolCallRequested::class, function (ToolCallRequested $event) use (&$decisions): void {
    $toolName = $event->getToolCall()->getName();

    if (isset($decisions[$toolName])) {
        if (!$decisions[$toolName]) {
            $event->deny('Tool previously denied by user.');
        }

        return;
    }

    echo \sprintf(
        "Allow tool '%s'? [y/N/always/never] ",
        $toolName
    );

    $input = strtolower(trim(fgets(\STDIN)));

    $allowed = \in_array($input, ['y', 'always'], true);

    if (\in_array($input, ['always', 'never'], true)) {
        $decisions[$toolName] = $allowed;
    }

    if (!$allowed) {
        $event->deny('User denied tool execution.');
    }
});

Step 7: Register the Listener in Symfony

In a Symfony application, the AsEventListener attribute registers the listener automatically — no service configuration needed. The event is inferred from the __invoke() argument:

1
2
3
4
5
6
7
8
9
10
11
12
13
namespace App\EventListener;

use Symfony\AI\Agent\Toolbox\Event\ToolCallRequested;
use Symfony\Component\EventDispatcher\Attribute\AsEventListener;

#[AsEventListener]
class ToolConfirmationListener
{
    public function __invoke(ToolCallRequested $event): void
    {
        // Your confirmation logic here
    }
}

Learn More

This work, including the code samples, is licensed under a Creative Commons BY-SA 3.0 license.
TOC
    Version