Tool Calling

Stevora supports LLM tool calling (function calling) with a multi-round execution loop, automatic argument parsing, and full tracing of every tool invocation. When a model decides it needs to call a tool, Stevora executes it, feeds the result back to the model, and continues until the model produces a final response.

Defining tools in a step

Tools are defined inline on the LLM step using the tools array. Each tool needs a name, description, and parameters object (JSON Schema format):

{
  "type": "llm",
  "name": "research-agent",
  "model": "gpt-4o",
  "systemPrompt": "You are a research assistant. Use the provided tools to answer questions.",
  "messages": [
    { "role": "user", "content": "{{state.question}}" }
  ],
  "tools": [
    {
      "name": "search_knowledge_base",
      "description": "Search the internal knowledge base for relevant documents",
      "parameters": {
        "type": "object",
        "properties": {
          "query": { "type": "string", "description": "The search query" },
          "limit": { "type": "number", "description": "Max results to return" }
        },
        "required": ["query"]
      }
    },
    {
      "name": "get_customer_data",
      "description": "Fetch customer details by ID",
      "parameters": {
        "type": "object",
        "properties": {
          "customerId": { "type": "string" }
        },
        "required": ["customerId"]
      }
    }
  ],
  "maxToolRounds": 5
}

The schema follows the llmToolDefSchema validation from llm.schema.ts:

const llmToolDefSchema = z.object({
  name: z.string().min(1),
  description: z.string(),
  parameters: z.record(z.unknown()),
});

Registering tool handlers

Tool definitions tell the model what tools are available. Tool handlers tell Stevora how to execute them. Register handlers using registerTool from the tool registry:

import { registerTool } from './modules/tool-tracing/tool-registry.js';

registerTool('search_knowledge_base', async (input) => {
  const query = input.query as string;
  const limit = (input.limit as number) ?? 10;
  const results = await knowledgeBase.search(query, limit);
  return { results };
});

registerTool('get_customer_data', async (input) => {
  const customer = await db.customers.findById(input.customerId as string);
  return { name: customer.name, email: customer.email, plan: customer.plan };
});

The ToolHandler type signature is:

type ToolHandler = (input: Record<string, unknown>) => Promise<unknown>;

Stevora automatically parses the JSON arguments string from the model into the input object. If parsing fails, the raw string is passed as { raw: tc.arguments }.

If a tool call references a name that has no registered handler, Stevora records the failure in the tool trace and returns an error message to the model so it can recover:

{ "error": "Tool 'unregistered_tool' not registered" }

Multi-round tool loop

When the model's response includes tool calls, Stevora enters a loop:

1. The model's response (with tool calls) is appended to the conversation as an assistant message
2. Each tool call is executed via its registered handler
3. Tool results are appended as tool messages with the corresponding toolCallId
4. The updated conversation is sent back to the model
5. If the model requests more tool calls, the loop repeats

This continues until the model returns a response without tool calls, or the maxToolRounds limit is reached.

// From llm-step-handler.ts -- the core loop
while (toolRounds <= (step.maxToolRounds ?? 10)) {
  const response = await provider.complete(request);

  if (response.toolCalls.length > 0 && step.tools && step.tools.length > 0) {
    toolRounds++;
    messages = [
      ...messages,
      {
        role: 'assistant',
        content: response.content ?? '',
        toolCalls: response.toolCalls,
      },
    ];

    const toolResults = await executeToolCalls(response.toolCalls, {
      llmCallId,
      stepRunId,
      workflowRunId: ctx.workflowRunId,
    });

    messages = [...messages, ...toolResults];
    continue;
  }

  finalResponse = response;
  break;
}

The default maxToolRounds is 10. You can set it from 1 to 20. If the model exhausts all rounds without producing a final answer, the step fails with the code MAX_TOOL_ROUNDS.

Tool tracing

Every tool invocation is persisted to the toolTrace table with full input/output recording. Each trace captures:

Successful tool executions also emit a TOOL_CALL_COMPLETED workflow event in the same database transaction, so tool activity shows up in the workflow event timeline.

This tracing is automatic -- you do not need to add any instrumentation to your tool handlers. Stevora wraps every call and records the result regardless of success or failure.

Error handling

Tool handlers can throw errors. When a handler throws, Stevora:

1. Records a failed tool trace with the error message
2. Returns the error to the model as a JSON string: { "error": "..." }
3. Continues the tool loop so the model can decide how to proceed

This means a single tool failure does not crash the step. The model receives the error and can retry, call a different tool, or produce a final answer based on the tools that did succeed.