ChatModelAgent

ChatModelAgent Overview

import "github.com/cloudwego/eino/adk"

What is ChatModelAgent

ChatModelAgent is the core Agent implementation of Eino ADK — it uses a ChatModel as the decision-maker, Tools as the action space, and autonomously drives problem-solving through a ReAct Loop.

For a complete introduction to ChatModelAgent concepts, the ReAct Loop, and the Middleware system, see: ChatModelAgent Introduction

ReAct Loop

When Tools are configured, ChatModelAgent executes in a ReAct loop:

Reason: Call the ChatModel, which decides the next action
Action: The model returns a ToolCall request
Act: Execute the corresponding Tool
Observation: Inject the Tool result into the context and start a new loop iteration

The loop continues until the model determines no further Tool calls are needed. Without Tools configured, it degrades to a single ChatModel invocation.

Configuration

TypedChatModelAgentConfig

type TypedChatModelAgentConfig[M MessageType] struct {
    Name        string
    Description string
    Instruction string

    Model       model.BaseModel[M]    // Required. Must support model.WithTools when using Tools

    ToolsConfig ToolsConfig
    GenModelInput TypedGenModelInput[M]

    Exit          tool.BaseTool         // NOT RECOMMENDED
    OutputKey     string                // NOT RECOMMENDED
    MaxIterations int                   // Default 20

    Handlers          []TypedChatModelAgentMiddleware[M]
    Middlewares        []AgentMiddleware  // Legacy compatibility

    ModelRetryConfig    *TypedModelRetryConfig[M]
    ModelFailoverConfig *ModelFailoverConfig[M]
}

// Default alias
type ChatModelAgentConfig = TypedChatModelAgentConfig[*schema.Message]

Field Descriptions

Field	Description
Name	Agent name. Required when used as AgentTool
Description	Agent capability description. Required when used as AgentTool
Instruction	System Prompt. Supports {Key} placeholders; default GenModelInput renders using SessionValues
Model	Required. Type model.BaseModel[M] ; must support model.WithTools when using Tools
ToolsConfig	Tool configuration, see below
GenModelInput	Custom input transformation. Default uses Instruction as System Message + f-string rendering
MaxIterations	Maximum ReAct loop iterations; exceeding this causes an error exit. Default 20
Handlers	Interface-style Middleware ( TypedChatModelAgentMiddleware[M] ), recommended
Middlewares	Struct-style Middleware ( AgentMiddleware ), legacy compatibility
ModelRetryConfig	Retry strategy for failed model calls
ModelFailoverConfig	Switch to backup model on failure. Requires configuring GetFailoverModel and ShouldFailover

💡 The default GenModelInput uses pyfmt rendering. { and } in Messages are treated as placeholders. To output these characters literally, escape them with {{ and }}.

ToolsConfig

type ToolsConfig struct {
    compose.ToolsNodeConfig

    ReturnDirectly     map[string]bool  // Tool names that return directly after invocation
    EmitInternalEvents bool             // Forward AgentTool internal events
}

ReturnDirectly: When a matching Tool completes execution, the Agent exits immediately without calling the model again. If multiple match, the first one is used
EmitInternalEvents: When a sub-Agent is invoked via AgentTool, its events are forwarded in real-time to the parent Agent’s event stream

Constructors

func NewChatModelAgent(ctx context.Context, config *ChatModelAgentConfig) (*ChatModelAgent, error)
func NewTypedChatModelAgent[M MessageType](ctx context.Context, config *TypedChatModelAgentConfig[M]) (*TypedChatModelAgent[M], error)

Middleware (ChatModelAgentMiddleware)

Interface Definition

type TypedChatModelAgentMiddleware[M MessageType] interface {
    BeforeAgent(ctx context.Context, runCtx *ChatModelAgentContext) (context.Context, *ChatModelAgentContext, error)
    AfterAgent(ctx context.Context, state *TypedChatModelAgentState[M]) (context.Context, error)

    BeforeModelRewriteState(ctx context.Context, state *TypedChatModelAgentState[M], mc *TypedModelContext[M]) (context.Context, *TypedChatModelAgentState[M], error)
    AfterModelRewriteState(ctx context.Context, state *TypedChatModelAgentState[M], mc *TypedModelContext[M]) (context.Context, *TypedChatModelAgentState[M], error)

    WrapModel(ctx context.Context, m model.BaseModel[M], mc *TypedModelContext[M]) (model.BaseModel[M], error)

    WrapInvokableToolCall(ctx context.Context, endpoint InvokableToolCallEndpoint, tCtx *ToolContext) (InvokableToolCallEndpoint, error)
    WrapStreamableToolCall(ctx context.Context, endpoint StreamableToolCallEndpoint, tCtx *ToolContext) (StreamableToolCallEndpoint, error)
    WrapEnhancedInvokableToolCall(ctx context.Context, endpoint EnhancedInvokableToolCallEndpoint, tCtx *ToolContext) (EnhancedInvokableToolCallEndpoint, error)
    WrapEnhancedStreamableToolCall(ctx context.Context, endpoint EnhancedStreamableToolCallEndpoint, tCtx *ToolContext) (EnhancedStreamableToolCallEndpoint, error)
}

type ChatModelAgentMiddleware = TypedChatModelAgentMiddleware[*schema.Message]

Embed *BaseChatModelAgentMiddleware to only override the methods you need:

type MyMiddleware struct {
    *adk.BaseChatModelAgentMiddleware
}

func (m *MyMiddleware) BeforeModelRewriteState(
    ctx context.Context,
    state *adk.ChatModelAgentState,
    mc *adk.ModelContext,
) (context.Context, *adk.ChatModelAgentState, error) {
    // Custom logic
    return ctx, state, nil
}

Hook Points

Hook	Timing	Modifiable Content
BeforeAgent	Before Agent runs (once only)	Instruction, Tools, ReturnDirectly, ToolSearchTool
AfterAgent	After Agent completes successfully	Read final state (no modification)
BeforeModelRewriteState	Before each model call	Messages, ToolInfos, DeferredToolInfos (persisted to state)
AfterModelRewriteState	After each model call	Messages (including model response), ToolInfos (persisted to state)
WrapModel	Wraps model invocation	Retry, failover, event emission (do not modify Messages)
WrapToolCall	Wraps tool invocation	Permission checks, logging, output rewriting

💡 The state returned by BeforeModelRewriteState is persisted by the framework to the agent’s internal state. Therefore, modifications in this hook (such as compressing Messages or filtering ToolInfos) will affect all subsequent iterations.

Core Types

ChatModelAgentContext (BeforeAgent Parameter)

type ChatModelAgentContext struct {
    Instruction    string
    Tools          []tool.BaseTool
    ReturnDirectly map[string]bool
    ToolSearchTool *schema.ToolInfo  // Model's native ToolSearch capability
}

ChatModelAgentState (BeforeModel/AfterModel Parameter)

type TypedChatModelAgentState[M MessageType] struct {
    Messages          []M
    ToolInfos         []*schema.ToolInfo         // Tool list passed to the model
    DeferredToolInfos []*schema.ToolInfo         // Server-side deferred retrieval tool list
}

type ChatModelAgentState = TypedChatModelAgentState[*schema.Message]

ModelContext (WrapModel Parameter)

type TypedModelContext[M MessageType] struct {
    Tools               []*schema.ToolInfo          // Deprecated: use state.ToolInfos
    ModelRetryConfig    *TypedModelRetryConfig[M]
    ModelFailoverConfig *ModelFailoverConfig[M]
}

type ModelContext = TypedModelContext[*schema.Message]

Execution Order

Model call chain (outer to inner):

AgentMiddleware.BeforeChatModel
BeforeModelRewriteState
failover wrapper (built-in)
retry wrapper (built-in)
event sender wrapper (built-in)
WrapModel (first registered = outermost)
callback injection (built-in)
Actual model call
AfterModelRewriteState
AgentMiddleware.AfterChatModel

Tool call chain (outer to inner):

event sender (built-in)
ToolsConfig.ToolCallMiddlewares
AgentMiddleware.WrapToolCall
WrapToolCall (first registered = outermost)
callback injection (built-in)
Actual tool call

AgentAsTool

Wrap a sub-Agent as a Tool so the parent Agent can invoke it autonomously via ToolCall:

subAgent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Name:        "researcher",
    Description: "Search and summarize information",
    Model:       chatModel,
    // ...
})

agentTool := adk.NewAgentTool(ctx, subAgent)

parentAgent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    // ...
    ToolsConfig: adk.ToolsConfig{
        ToolsNodeConfig: compose.ToolsNodeConfig{
            Tools: []tool.BaseTool{agentTool},
        },
    },
})

Generic version: adk.NewTypedAgentTool[M](ctx, agent, options...)

Options: WithFullChatHistoryAsInput() (pass complete chat history), WithAgentInputSchema(schema) (custom input schema)

ModelRetry

When configured, ChatModel calls are automatically retried on failure. When an error occurs during a streaming response, the current stream is still returned via AgentEvent, and consuming the MessageStream yields a WillRetryError:

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    // ...
    ModelRetryConfig: &adk.ModelRetryConfig{
        // Retry strategy configuration
    },
})

// Handle WillRetryError when consuming the event stream
stream := event.Output.MessageOutput.MessageStream
for {
    msg, err := stream.Recv()
    if err == io.EOF {
        break
    }
    if err != nil {
        var willRetry *adk.WillRetryError
        if errors.As(err, &willRetry) {
            log.Printf("Attempt %d failed, retrying...", willRetry.RetryAttempt)
            break // Wait for next event
        }
        break
    }
    displayChunk(msg)
}

ModelFailover

When configured, the agent switches to a backup model on failure:

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Model: primaryModel,
    ModelFailoverConfig: &adk.ModelFailoverConfig{
        GetFailoverModel: func(ctx context.Context, err error) (model.BaseModel[*schema.Message], error) {
            return backupModel, nil
        },
        ShouldFailover: func(err error) bool {
            return true // Decide whether to failover based on error type
        },
    },
})

Cancel

New runtime cancellation capability added in v0.9. See Agent Cancel and TurnLoop for details.

cancelOpt, cancelFn := adk.WithCancel()
iter := runner.Run(ctx, messages, cancelOpt)

// Cancel later (CancelMode supports bitmask combinations)
handle := cancelFn(adk.CancelAfterChatModel | adk.CancelAfterToolCalls)
handle.Wait() // Wait for cancellation to complete

ChatModel Failover Guide

Feedback

Was this page helpful?

Please tell us how we can improve.

Last modified May 21, 2026: fix(docs): update broken links to agent_cancel_and_turnloop_quickstart (88d7f40a)