Building Multi-Agent Systems: A Complete Guide with Email Triage Example

Introduction

Multi-agent systems represent a powerful approach to solving complex problems by breaking them down into smaller, specialized components. A multiagent system (MAS) consists of multiple artificial intelligence (AI) agents working collectively to perform tasks on behalf of a user or another system.

In this guide, we'll explore what multi-agent systems are, why they're useful, and walk through building a practical email triage system using LangChain.js and LangGraph.

What Are Multi-Agent Systems?

Each agent can have its own prompt, LLM, tools, and other custom code to best collaborate with the other agents. Think of it like having a team of specialists, where each expert focuses on their domain of expertise rather than one generalist trying to handle everything.

Key Benefits of Multi-Agent Systems

Grouping tools/responsibilities can give better results. An agent is more likely to succeed on a focused task than if it has to select from dozens of tools. Here are the main advantages:

Modularity: Separate agents make it easier to develop, test, and maintain complex systems. You can update one agent without affecting others.

Specialization: Each agent becomes an expert in its domain, leading to better performance than a single general-purpose agent.

Scalability: Multiagent systems can adjust to varying environments by adding, removing or adapting agents.

Control: You have explicit control over how agents communicate and when they're activated.

Multi-Agent Architectures

There are several ways to connect agents in a multi-agent system:

1. Supervisor Architecture

Each agent communicates with a single supervisor agent. Supervisor agent makes decisions on which agent should be called next. This is perfect for our email triage system where we need centralized decision-making.

2. Network Architecture

Each agent can communicate with every other agent. Any agent can decide which other agent to call next. This works well when there's no clear hierarchy.

3. Hierarchical Architecture

You can define a multi-agent system with a supervisor of supervisors. This enables complex, nested team structures.

Building an Email Triage System

Let's build a practical multi-agent email triage system that automatically categorizes and routes incoming emails to the right departments. Our system will use the supervisor architecture with specialized agents for different email types.

System Architecture

Our email triage system consists of:

Supervisor Agent: Orchestrates the triage process and makes final routing decisions
Customer Support Agent: Handles technical issues and customer inquiries
Sales Agent: Manages business inquiries and lead qualification
HR Agent: Processes recruitment and employee-related emails
Spam Filter Agent: Identifies and filters low-priority communications

Step 1: Project Setup

First, let's set up our TypeScript project with the necessary dependencies:

bash

1mkdir multi-agent-email-triage
2cd multi-agent-email-triage
3npm init -y

Install the required packages:

bash

1npm install @langchain/core @langchain/langgraph @langchain/openai dotenv zod
2npm install -D typescript @types/node ts-node

Create a tsconfig.json:

json

1{
2  "compilerOptions": {
3    "target": "ES2020",
4    "module": "CommonJS", 
5    "lib": ["ES2020"],
6    "outDir": "./dist",
7    "rootDir": "./src",
8    "strict": true,
9    "esModuleInterop": true,
10    "skipLibCheck": true,
11    "forceConsistentCasingInFileNames": true,
12    "resolveJsonModule": true
13  },
14  "include": ["src/**/*"],
15  "exclude": ["node_modules", "dist"]
16}

Step 2: Define Types

Create src/types.ts to define our data structures:

typescript

1export interface Email {
2  id: string;
3  from: string;
4  to: string;
5  subject: string;
6  body: string;
7  timestamp: Date;
8  priority?: 'low' | 'medium' | 'high' | 'urgent';
9  category?: string;
10}
11
12export interface TriageResult {
13  emailId: string;
14  category: string;
15  priority: 'low' | 'medium' | 'high' | 'urgent';
16  assignedAgent: string;
17  summary: string;
18  suggestedActions: string[];
19  requiresHumanReview: boolean;
20}
21
22export interface AgentDecision {
23  shouldHandle: boolean;
24  confidence: number;
25  reasoning: string;
26  suggestedActions?: string[];
27  escalate?: boolean;
28}

Step 3: Implement Base Agent

Create src/agents/baseAgent.ts:

typescript

1import { Email, AgentDecision } from '../types';
2import { ChatOpenAI } from '@langchain/openai';
3import { PromptTemplate } from '@langchain/core/prompts';
4
5export abstract class BaseAgent {
6  protected name: string;
7  protected llm: ChatOpenAI;
8  protected prompt: PromptTemplate;
9
10  constructor(name: string) {
11    this.name = name;
12    this.llm = new ChatOpenAI({
13      modelName: 'gpt-4o-mini',
14      temperature: 0.1,
15    });
16  }
17
18  abstract evaluate(email: Email): Promise<AgentDecision>;
19
20  protected async generateDecision(email: Email, systemPrompt: string): Promise<AgentDecision> {
21    try {
22      const prompt = `${systemPrompt}
23
24Email Details:
25From: ${email.from}
26Subject: ${email.subject}
27Body: ${email.body}
28
29Respond with JSON containing:
30- shouldHandle: boolean
31- confidence: number (0-100)
32- reasoning: string
33- suggestedActions: string[]
34- escalate: boolean`;
35
36      const response = await this.llm.invoke([{ role: 'user', content: prompt }]);
37      return JSON.parse(response.content as string);
38    } catch (error) {
39      console.error(`Error in ${this.name} agent:`, error);
40      return {
41        shouldHandle: false,
42        confidence: 0,
43        reasoning: 'Agent error - requires manual review',
44        escalate: true
45      };
46    }
47  }
48}

Step 4: Implement Specialized Agents

Create src/agents/customerSupportAgent.ts:

typescript

1import { BaseAgent } from './baseAgent';
2import { Email, AgentDecision } from '../types';
3
4export class CustomerSupportAgent extends BaseAgent {
5  constructor() {
6    super('CustomerSupport');
7  }
8
9  async evaluate(email: Email): Promise<AgentDecision> {
10    const systemPrompt = `You are a Customer Support specialist. Evaluate if this email requires customer support attention.
11
12Look for:
13- Technical issues, bugs, error messages
14- Account access problems
15- Feature questions or how-to requests
16- Product complaints or feedback
17- Billing or subscription issues
18
19Consider the urgency and complexity of the issue.`;
20
21    return this.generateDecision(email, systemPrompt);
22  }
23}

Create src/agents/salesAgent.ts:

typescript

1import { BaseAgent } from './baseAgent';
2import { Email, AgentDecision } from '../types';
3
4export class SalesAgent extends BaseAgent {
5  constructor() {
6    super('Sales');
7  }
8
9  async evaluate(email: Email): Promise<AgentDecision> {
10    const systemPrompt = `You are a Sales specialist. Evaluate if this email represents a sales opportunity.
11
12Look for:
13- Pricing inquiries
14- Demo or trial requests
15- Partnership opportunities
16- Enterprise or bulk purchase interest
17- Competitor comparisons
18- Budget discussions
19
20Assess the lead quality and urgency.`;
21
22    return this.generateDecision(email, systemPrompt);
23  }
24}

Create src/agents/hrAgent.ts:

typescript

1import { BaseAgent } from './baseAgent';
2import { Email, AgentDecision } from '../types';
3
4export class HRAgent extends BaseAgent {
5  constructor() {
6    super('HR');
7  }
8
9  async evaluate(email: Email): Promise<AgentDecision> {
10    const systemPrompt = `You are an HR specialist. Evaluate if this email requires HR attention.
11
12Look for:
13- Job applications and resumes
14- Interview scheduling
15- Employee inquiries
16- Policy questions
17- Benefits or payroll issues
18- Recruitment outreach
19
20Determine if immediate HR attention is needed.`;
21
22    return this.generateDecision(email, systemPrompt);
23  }
24}

Create src/agents/spamFilterAgent.ts:

typescript

1import { BaseAgent } from './baseAgent';
2import { Email, AgentDecision } from '../types';
3
4export class SpamFilterAgent extends BaseAgent {
5  constructor() {
6    super('SpamFilter');
7  }
8
9  async evaluate(email: Email): Promise<AgentDecision> {
10    const systemPrompt = `You are a Spam Filter specialist. Evaluate if this email is spam or low-priority.
11
12Look for:
13- Marketing emails from unknown senders
14- Phishing attempts
15- Automated newsletters unrelated to business
16- Suspicious links or attachments
17- Generic mass-sent content
18- Irrelevant promotional content
19
20Be conservative - when in doubt, don't mark as spam.`;
21
22    return this.generateDecision(email, systemPrompt);
23  }
24}

Step 5: Implement the Supervisor Agent

Create src/agents/supervisorAgent.ts:

typescript

1import { Email, TriageResult, AgentDecision } from '../types';
2import { CustomerSupportAgent } from './customerSupportAgent';
3import { SalesAgent } from './salesAgent';
4import { HRAgent } from './hrAgent';
5import { SpamFilterAgent } from './spamFilterAgent';
6
7export class SupervisorAgent {
8  private customerSupportAgent: CustomerSupportAgent;
9  private salesAgent: SalesAgent;
10  private hrAgent: HRAgent;
11  private spamFilterAgent: SpamFilterAgent;
12
13  constructor() {
14    this.customerSupportAgent = new CustomerSupportAgent();
15    this.salesAgent = new SalesAgent();
16    this.hrAgent = new HRAgent();
17    this.spamFilterAgent = new SpamFilterAgent();
18  }
19
20  async triageEmail(email: Email): Promise<TriageResult> {
21    // First, check if it's spam (quick elimination)
22    const spamDecision = await this.spamFilterAgent.evaluate(email);
23    if (spamDecision.shouldHandle && spamDecision.confidence > 70) {
24      return this.createTriageResult(email, 'spam', 'low', 'SpamFilter', spamDecision);
25    }
26
27    // Run all specialist agents in parallel
28    const [supportDecision, salesDecision, hrDecision] = await Promise.all([
29      this.customerSupportAgent.evaluate(email),
30      this.salesAgent.evaluate(email),
31      this.hrAgent.evaluate(email)
32    ]);
33
34    // Determine the best agent based on confidence scores
35    const decisions = [
36      { agent: 'CustomerSupport', decision: supportDecision },
37      { agent: 'Sales', decision: salesDecision },
38      { agent: 'HR', decision: hrDecision }
39    ];
40
41    // Filter agents that want to handle the email
42    const interestedAgents = decisions.filter(d => d.decision.shouldHandle);
43
44    if (interestedAgents.length === 0) {
45      // No agent wants to handle - default to manual review
46      return this.createTriageResult(email, 'unclassified', 'medium', 'Manual', {
47        shouldHandle: true,
48        confidence: 0,
49        reasoning: 'No agent could confidently categorize this email',
50        escalate: true
51      });
52    }
53
54    // Select the agent with highest confidence
55    const selectedAgent = interestedAgents.reduce((prev, current) => 
56      current.decision.confidence > prev.decision.confidence ? current : prev
57    );
58
59    const priority = this.determinePriority(selectedAgent.decision);
60    const category = this.determineCategory(selectedAgent.agent, selectedAgent.decision);
61
62    return this.createTriageResult(email, category, priority, selectedAgent.agent, selectedAgent.decision);
63  }
64
65  private createTriageResult(
66    email: Email, 
67    category: string, 
68    priority: 'low' | 'medium' | 'high' | 'urgent',
69    assignedAgent: string, 
70    decision: AgentDecision
71  ): TriageResult {
72    return {
73      emailId: email.id,
74      category,
75      priority,
76      assignedAgent,
77      summary: decision.reasoning,
78      suggestedActions: decision.suggestedActions || [],
79      requiresHumanReview: decision.escalate || false
80    };
81  }
82
83  private determinePriority(decision: AgentDecision): 'low' | 'medium' | 'high' | 'urgent' {
84    if (decision.escalate) return 'urgent';
85    if (decision.confidence > 80) return 'high';
86    if (decision.confidence > 60) return 'medium';
87    return 'low';
88  }
89
90  private determineCategory(agent: string, decision: AgentDecision): string {
91    const baseCategories = {
92      'CustomerSupport': 'support',
93      'Sales': 'sales',
94      'HR': 'hr',
95      'SpamFilter': 'spam'
96    };
97
98    return baseCategories[agent] || 'unclassified';
99  }
100}

Step 6: Create the Main Application

Create src/emailTriageSystem.ts:

typescript

1import { SupervisorAgent } from './agents/supervisorAgent';
2import { Email, TriageResult } from './types';
3
4export class EmailTriageSystem {
5  private supervisor: SupervisorAgent;
6
7  constructor() {
8    this.supervisor = new SupervisorAgent();
9  }
10
11  async processEmail(email: Email): Promise<TriageResult> {
12    const startTime = Date.now();
13    
14    try {
15      const result = await this.supervisor.triageEmail(email);
16      
17      // Log metrics for monitoring
18      console.log(JSON.stringify({
19        timestamp: new Date().toISOString(),
20        emailId: email.id,
21        assignedAgent: result.assignedAgent,
22        priority: result.priority,
23        processingTime: Date.now() - startTime,
24        category: result.category
25      }));
26
27      return result;
28    } catch (error) {
29      console.error('Error processing email:', error);
30      throw error;
31    }
32  }
33
34  async processBatch(emails: Email[]): Promise<TriageResult[]> {
35    return Promise.all(emails.map(email => this.processEmail(email)));
36  }
37}

Step 7: Example Usage

Create src/example.ts:

typescript

1import { EmailTriageSystem } from './emailTriageSystem';
2import { Email } from './types';
3
4async function main() {
5  const triageSystem = new EmailTriageSystem();
6
7  // Example emails
8  const emails: Email[] = [
9    {
10      id: '1',
11      from: 'customer@example.com',
12      to: 'support@company.com',
13      subject: 'Cannot login to my account',
14      body: 'I keep getting error 500 when trying to log in. This has been happening for 2 days.',
15      timestamp: new Date()
16    },
17    {
18      id: '2',
19      from: 'prospect@bigcorp.com',
20      to: 'sales@company.com',
21      subject: 'Enterprise pricing inquiry',
22      body: 'We are interested in your enterprise plan for 500+ users. Could you send pricing?',
23      timestamp: new Date()
24    },
25    {
26      id: '3',
27      from: 'candidate@email.com',
28      to: 'jobs@company.com',
29      subject: 'Application for Software Engineer Position',
30      body: 'Please find attached my resume for the senior software engineer role.',
31      timestamp: new Date()
32    }
33  ];
34
35  // Process emails
36  for (const email of emails) {
37    const result = await triageSystem.processEmail(email);
38    console.log(`\nEmail ${email.id}:`);
39    console.log(`Category: ${result.category}`);
40    console.log(`Priority: ${result.priority}`);
41    console.log(`Assigned to: ${result.assignedAgent}`);
42    console.log(`Summary: ${result.summary}`);
43    console.log(`Actions: ${result.suggestedActions.join(', ')}`);
44  }
45}
46
47main().catch(console.error);

Benefits Over Single-Agent Systems

Improved Accuracy

Specialist agents consistently outperform generalist agents in their domains. A dedicated Sales Agent better identifies qualified leads than a general-purpose agent trying to handle all email types.

Better Scalability

Adding new capabilities doesn't require retraining existing agents. You simply add new specialists to the system.

Fault Tolerance

If one agent fails or performs poorly, others can continue working. The system degrades gracefully rather than failing completely.

Explainable Decisions

Each agent provides reasoning for its decisions, making the overall system more transparent and debuggable.

Performance Considerations

Parallel Processing

Running agent evaluations in parallel significantly reduces response time:

typescript

1// Parallel evaluation reduces latency
2const [supportDecision, salesDecision, hrDecision] = await Promise.all([
3  this.customerSupportAgent.evaluate(email),
4  this.salesAgent.evaluate(email), 
5  this.hrAgent.evaluate(email)
6]);

Cost Optimization

The spam filter runs first to quickly eliminate low-value emails before expensive LLM evaluations:

typescript

1// Quick spam check before expensive evaluations
2const spamDecision = await this.spamFilterAgent.evaluate(email);
3if (spamDecision.shouldHandle && spamDecision.confidence > 70) {
4  return this.filterAsSpam(email);
5}

Caching and Memoization

Similar emails can reuse previous evaluations to reduce API calls and improve response times.

Extending the System

Adding New Agents

To add a Legal Agent:

typescript

1export class LegalAgent extends BaseAgent {
2  constructor() {
3    super('Legal');
4  }
5
6  async evaluate(email: Email): Promise<AgentDecision> {
7    const systemPrompt = `You are a Legal specialist. Evaluate if this email requires legal attention.
8
9Look for:
10- Contract discussions and negotiations
11- Legal notices and compliance issues
12- Intellectual property matters
13- Privacy and data protection concerns
14- Litigation or dispute-related content
15- Regulatory compliance questions
16
17Assess urgency and legal risk level.`;
18
19    return this.generateDecision(email, systemPrompt);
20  }
21}

Then add it to the supervisor:

typescript

1constructor() {
2  // ... existing agents
3  this.legalAgent = new LegalAgent();
4}
5
6async triageEmail(email: Email): Promise<TriageResult> {
7  // Add to parallel evaluation
8  const [supportDecision, salesDecision, hrDecision, legalDecision] = await Promise.all([
9    this.customerSupportAgent.evaluate(email),
10    this.salesAgent.evaluate(email),
11    this.hrAgent.evaluate(email),
12    this.legalAgent.evaluate(email)
13  ]);
14}

Custom Routing Logic

Implement domain-specific routing rules:

typescript

1private async makeRoutingDecision(email: Email, decisions: any[]): Promise<TriageResult> {
2  // Custom business logic
3  if (this.isVIPCustomer(email.from)) {
4    return this.routeToVIPSupport(email);
5  }
6  
7  if (this.isHighValueProspect(email)) {
8    return this.routeToSeniorSales(email);
9  }
10  
11  // Standard routing logic...
12  return this.standardRouting(decisions);
13}
14
15private isVIPCustomer(emailAddress: string): boolean {
16  const vipDomains = ['bigclient.com', 'enterprise.com'];
17  return vipDomains.some(domain => emailAddress.includes(domain));
18}
19
20private isHighValueProspect(email: Email): boolean {
21  const indicators = ['enterprise', 'bulk', '1000+', 'enterprise pricing'];
22  return indicators.some(indicator => 
23    email.subject.toLowerCase().includes(indicator) || 
24    email.body.toLowerCase().includes(indicator)
25  );
26}

Integration with External Systems

Connect to your existing tools:

typescript

1async triageEmail(email: Email): Promise<TriageResult> {
2  const result = await this.supervisor.triageEmail(email);
3  
4  // Create ticket in helpdesk system
5  if (result.assignedAgent === 'CustomerSupport') {
6    await this.createSupportTicket(email, result);
7  }
8  
9  // Add lead to CRM
10  if (result.assignedAgent === 'Sales') {
11    await this.createCRMLead(email, result);
12  }
13  
14  // Notify HR system
15  if (result.assignedAgent === 'HR') {
16    await this.notifyHRSystem(email, result);
17  }
18  
19  return result;
20}
21
22private async createSupportTicket(email: Email, result: TriageResult): Promise<void> {
23  // Integration with helpdesk API
24  const ticket = {
25    subject: email.subject,
26    description: email.body,
27    priority: result.priority,
28    customerEmail: email.from,
29    category: result.category
30  };
31  
32  // await helpdeskAPI.createTicket(ticket);
33  console.log('Support ticket created:', ticket);
34}
35
36private async createCRMLead(email: Email, result: TriageResult): Promise<void> {
37  // Integration with CRM API
38  const lead = {
39    email: email.from,
40    source: 'email',
41    priority: result.priority,
42    notes: result.summary,
43    suggestedActions: result.suggestedActions
44  };
45  
46  // await crmAPI.createLead(lead);
47  console.log('CRM lead created:', lead);
48}

Other Use Cases for Multi-Agent Systems

Multi-agent architectures excel in many domains beyond email triage:

Content Moderation

Toxicity Agent: Detects harmful language and harassment
Spam Agent: Identifies promotional and irrelevant content
Policy Agent: Checks against community guidelines
Context Agent: Analyzes cultural and situational appropriateness

Financial Analysis

Risk Agent: Assesses investment and credit risks
Fraud Agent: Detects suspicious transactions and patterns
Compliance Agent: Ensures regulatory adherence
Market Agent: Analyzes trends and opportunities

Healthcare Triage

Urgency Agent: Determines medical priority levels
Specialty Agent: Routes to appropriate medical departments
Symptom Agent: Analyzes patient-reported symptoms
Protocol Agent: Suggests care pathways and treatments

Legal Document Review

Contract Agent: Analyzes agreements and terms
Risk Agent: Identifies legal liabilities and concerns
Compliance Agent: Checks regulatory requirements
Classification Agent: Categorizes by practice area and urgency

Testing and Validation

Unit Testing Agents

Test individual agent decisions:

typescript

1describe('CustomerSupportAgent', () => {
2  let agent: CustomerSupportAgent;
3
4  beforeEach(() => {
5    agent = new CustomerSupportAgent();
6  });
7
8  it('should identify technical issues', async () => {
9    const email: Email = {
10      id: 'test-1',
11      from: 'user@example.com',
12      to: 'support@company.com',
13      subject: 'Login Error - Cannot Access Account',
14      body: 'Getting error code 500 when trying to log in. This started yesterday.',
15      timestamp: new Date()
16    };
17    
18    const decision = await agent.evaluate(email);
19    expect(decision.shouldHandle).toBe(true);
20    expect(decision.confidence).toBeGreaterThan(80);
21    expect(decision.reasoning).toContain('technical');
22  });
23
24  it('should not handle sales inquiries', async () => {
25    const email: Email = {
26      id: 'test-2',
27      from: 'prospect@company.com',
28      to: 'contact@company.com',
29      subject: 'Pricing Information Request',
30      body: 'Could you send me pricing for your premium plan?',
31      timestamp: new Date()
32    };
33    
34    const decision = await agent.evaluate(email);
35    expect(decision.shouldHandle).toBe(false);
36  });
37});

Integration Testing

Test the complete triage flow:

typescript

1describe('Email Triage System', () => {
2  let triageSystem: EmailTriageSystem;
3
4  beforeEach(() => {
5    triageSystem = new EmailTriageSystem();
6  });
7
8  it('should route support emails correctly', async () => {
9    const supportEmail: Email = {
10      id: 'integration-1',
11      from: 'customer@example.com',
12      to: 'help@company.com',
13      subject: 'Bug Report - Data Not Saving',
14      body: 'When I click save, nothing happens. Please help!',
15      timestamp: new Date()
16    };
17
18    const result = await triageSystem.processEmail(supportEmail);
19    expect(result.assignedAgent).toBe('CustomerSupport');
20    expect(result.priority).toBeOneOf(['medium', 'high']);
21    expect(result.category).toBe('support');
22  });
23
24  it('should handle batch processing', async () => {
25    const emails: Email[] = [
26      createTestEmail('1', 'support@company.com', 'Bug report'),
27      createTestEmail('2', 'sales@company.com', 'Pricing inquiry'),
28      createTestEmail('3', 'hr@company.com', 'Job application')
29    ];
30
31    const results = await triageSystem.processBatch(emails);
32    expect(results).toHaveLength(3);
33    expect(results.every(r => r.assignedAgent)).toBe(true);
34  });
35});
36
37function createTestEmail(id: string, to: string, subject: string): Email {
38  return {
39    id,
40    from: `test${id}@example.com`,
41    to,
42    subject,
43    body: `Test email body for ${subject}`,
44    timestamp: new Date()
45  };
46}

A/B Testing

Compare multi-agent performance against single-agent systems:

typescript

1async function comparePerformance() {
2  const testEmails = await loadTestDataset();
3  
4  // Run both systems
5  const multiAgentResults = await Promise.all(
6    testEmails.map(email => multiAgentSupervisor.triageEmail(email))
7  );
8  
9  const singleAgentResults = await Promise.all(
10    testEmails.map(email => singleAgent.triageEmail(email))
11  );
12
13  // Calculate metrics
14  const multiAgentAccuracy = calculateAccuracy(multiAgentResults, testEmails);
15  const singleAgentAccuracy = calculateAccuracy(singleAgentResults, testEmails);
16
17  console.log('Performance Comparison:');
18  console.log(`Multi-Agent Accuracy: ${multiAgentAccuracy}%`);
19  console.log(`Single-Agent Accuracy: ${singleAgentAccuracy}%`);
20  
21  // Track costs and latency
22  await recordMetrics({
23    multiAgent: {
24      accuracy: multiAgentAccuracy,
25      avgLatency: calculateAverageLatency(multiAgentResults),
26      cost: calculateCost(multiAgentResults)
27    },
28    singleAgent: {
29      accuracy: singleAgentAccuracy,
30      avgLatency: calculateAverageLatency(singleAgentResults),
31      cost: calculateCost(singleAgentResults)
32    }
33  });
34}

Production Deployment

Environment Configuration

Use environment variables for different deployment stages:

typescript

1// src/config.ts
2export const config = {
3  llm: {
4    model: process.env.NODE_ENV === 'production' ? 'gpt-4' : 'gpt-4o-mini',
5    temperature: parseFloat(process.env.LLM_TEMPERATURE || '0.1'),
6    maxRetries: parseInt(process.env.MAX_RETRIES || '3'),
7    timeout: parseInt(process.env.LLM_TIMEOUT || '30000')
8  },
9  
10  agents: {
11    enableParallelProcessing: process.env.ENABLE_PARALLEL === 'true',
12    confidenceThreshold: parseFloat(process.env.CONFIDENCE_THRESHOLD || '0.7'),
13    maxConcurrentRequests: parseInt(process.env.MAX_CONCURRENT || '10')
14  },
15  
16  monitoring: {
17    logLevel: process.env.LOG_LEVEL || 'info',
18    enableMetrics: process.env.ENABLE_METRICS === 'true'
19  }
20};

Monitoring and Alerting

Track system performance:

typescript

1class MetricsCollector {
2  private metrics: Map<string, any[]> = new Map();
3
4  recordTriageMetrics(email: Email, result: TriageResult, processingTime: number) {
5    const metric = {
6      timestamp: new Date().toISOString(),
7      emailId: email.id,
8      assignedAgent: result.assignedAgent,
9      confidence: result.confidence,
10      processingTime,
11      priority: result.priority,
12      category: result.category
13    };
14
15    console.log(JSON.stringify(metric));
16    
17    // Store for analysis
18    if (!this.metrics.has('triageResults')) {
19      this.metrics.set('triageResults', []);
20    }
21    this.metrics.get('triageResults')!.push(metric);
22
23    // Alert on anomalies
24    if (processingTime > 10000) {
25      this.sendAlert('HIGH_LATENCY', `Processing time: ${processingTime}ms`);
26    }
27  }
28
29  private sendAlert(type: string, message: string) {
30    console.error(`ALERT [${type}]: ${message}`);
31    // Integration with alerting system
32  }
33
34  getMetricsSummary() {
35    const results = this.metrics.get('triageResults') || [];
36    return {
37      totalProcessed: results.length,
38      averageProcessingTime: results.reduce((sum, r) => sum + r.processingTime, 0) / results.length,
39      agentDistribution: this.calculateAgentDistribution(results),
40      priorityDistribution: this.calculatePriorityDistribution(results)
41    };
42  }
43
44  private calculateAgentDistribution(results: any[]) {
45    return results.reduce((dist, result) => {
46      dist[result.assignedAgent] = (dist[result.assignedAgent] || 0) + 1;
47      return dist;
48    }, {});
49  }
50
51  private calculatePriorityDistribution(results: any[]) {
52    return results.reduce((dist, result) => {
53      dist[result.priority] = (dist[result.priority] || 0) + 1;
54      return dist;
55    }, {});
56  }
57}

Error Handling and Resilience

Implement robust error handling:

typescript

1export class ResilientBaseAgent extends BaseAgent {
2  private retryCount = 0;
3  private maxRetries = 3;
4
5  async evaluate(email: Email): Promise<AgentDecision> {
6    for (let attempt = 1; attempt <= this.maxRetries; attempt++) {
7      try {
8        return await this.performEvaluationWithTimeout(email);
9      } catch (error) {
10        console.error(`Attempt ${attempt} failed for ${this.name} agent:`, error);
11        
12        if (attempt === this.maxRetries) {
13          // Final fallback
14          return this.createFallbackDecision(error);
15        }
16        
17        // Exponential backoff
18        await this.delay(Math.pow(2, attempt) * 1000);
19      }
20    }
21    
22    // TypeScript satisfaction - this won't be reached
23    return this.createFallbackDecision(new Error('Max retries exceeded'));
24  }
25
26  private async performEvaluationWithTimeout(email: Email): Promise<AgentDecision> {
27    return Promise.race([
28      this.performEvaluation(email),
29      this.createTimeoutPromise()
30    ]);
31  }
32
33  private createTimeoutPromise(): Promise<AgentDecision> {
34    return new Promise((_, reject) => {
35      setTimeout(() => reject(new Error('Evaluation timeout')), 30000);
36    });
37  }
38
39  private createFallbackDecision(error: Error): AgentDecision {
40    return {
41      shouldHandle: false,
42      confidence: 0,
43      reasoning: `Agent error (${error.message}) - requires manual review`,
44      escalate: true,
45      suggestedActions: ['Manual review required', 'Check system logs']
46    };
47  }
48
49  private delay(ms: number): Promise<void> {
50    return new Promise(resolve => setTimeout(resolve, ms));
51  }
52}

Cost Optimization Strategies

Intelligent Caching

Cache similar email patterns:

typescript

1class TriageCache {
2  private cache = new Map<string, { result: AgentDecision; timestamp: number }>();
3  private readonly TTL = 24 * 60 * 60 * 1000; // 24 hours
4
5  generateCacheKey(email: Email): string {
6    // Create hash based on subject patterns and sender domain
7    const domain = email.from.split('@')[1];
8    const subjectPattern = email.subject.toLowerCase()
9      .replace(/\d+/g, 'NUM')
10      .replace(/[^\w\s]/g, '');
11    
12    return `${domain}:${subjectPattern}`;
13  }
14
15  get(key: string): AgentDecision | null {
16    const cached = this.cache.get(key);
17    if (!cached) return null;
18
19    // Check if expired
20    if (Date.now() - cached.timestamp > this.TTL) {
21      this.cache.delete(key);
22      return null;
23    }
24
25    return cached.result;
26  }
27
28  set(key: string, result: AgentDecision): void {
29    this.cache.set(key, {
30      result,
31      timestamp: Date.now()
32    });
33  }
34
35  clear(): void {
36    this.cache.clear();
37  }
38}

Tiered Model Usage

Use cheaper models for initial filtering:

typescript

1class TieredEvaluationAgent extends BaseAgent {
2  private quickClassifier: ChatOpenAI;
3  private detailedAnalyzer: ChatOpenAI;
4
5  constructor(name: string) {
6    super(name);
7    
8    // Cheaper model for quick classification
9    this.quickClassifier = new ChatOpenAI({
10      modelName: 'gpt-4o-mini',
11      temperature: 0.1,
12    });
13    
14    // More expensive model for detailed analysis
15    this.detailedAnalyzer = new ChatOpenAI({
16      modelName: 'gpt-4',
17      temperature: 0.1,
18    });
19  }
20
21  async evaluate(email: Email): Promise<AgentDecision> {
22    // Quick classification first
23    const quickDecision = await this.quickClassification(email);
24    
25    // If confidence is high enough, use quick decision
26    if (quickDecision.confidence > 85) {
27      return quickDecision;
28    }
29    
30    // Otherwise, use detailed analysis
31    return this.detailedAnalysis(email);
32  }
33
34  private async quickClassification(email: Email): Promise<AgentDecision> {
35    const prompt = `Quick classification: Is this email relevant to ${this.name}? 
36    Email: ${email.subject} - ${email.body.substring(0, 200)}
37    Respond with JSON: {shouldHandle: boolean, confidence: number}`;
38    
39    const response = await this.quickClassifier.invoke([{ role: 'user', content: prompt }]);
40    return JSON.parse(response.content as string);
41  }
42
43  private async detailedAnalysis(email: Email): Promise<AgentDecision> {
44    // Full detailed analysis with more expensive model
45    return this.generateDecision(email, this.getDetailedPrompt());
46  }
47
48  protected abstract getDetailedPrompt(): string;
49}

Batch Processing

Process multiple emails together when possible:

typescript

1export class BatchTriageSystem extends EmailTriageSystem {
2  async processBatchOptimized(emails: Email[]): Promise<TriageResult[]> {
3    // Group similar emails for batch processing
4    const batches = this.groupSimilarEmails(emails);
5    
6    // Process each batch with optimizations
7    const results = await Promise.all(
8      batches.map(batch => this.processBatchGroup(batch))
9    );
10    
11    return results.flat();
12  }
13
14  private groupSimilarEmails(emails: Email[]): Email[][] {
15    const groups = new Map<string, Email[]>();
16    
17    emails.forEach(email => {
18      const pattern = this.extractPattern(email);
19      if (!groups.has(pattern)) {
20        groups.set(pattern, []);
21      }
22      groups.get(pattern)!.push(email);
23    });
24    
25    return Array.from(groups.values());
26  }
27
28  private extractPattern(email: Email): string {
29    const domain = email.from.split('@')[1];
30    const hasAttachment = email.body.includes('attach');
31    const isUrgent = /urgent|asap|immediate/i.test(email.subject + email.body);
32    
33    return `${domain}:${hasAttachment}:${isUrgent}`;
34  }
35
36  private async processBatchGroup(emails: Email[]): Promise<TriageResult[]> {
37    if (emails.length === 1) {
38      return [await this.processEmail(emails[0])];
39    }
40
41    // For similar emails, process first one fully, then apply pattern to others
42    const [firstEmail, ...otherEmails] = emails;
43    const firstResult = await this.processEmail(firstEmail);
44    
45    // Apply similar logic to other emails with adjustments
46    const otherResults = await Promise.all(
47      otherEmails.map(email => this.processEmailWithPattern(email, firstResult))
48    );
49    
50    return [firstResult, ...otherResults];
51  }
52
53  private async processEmailWithPattern(email: Email, pattern: TriageResult): Promise<TriageResult> {
54    // Use pattern as guidance but still validate
55    if (this.isSimilarEnough(email, pattern)) {
56      return {
57        ...pattern,
58        emailId: email.id,
59        summary: `Similar to ${pattern.emailId}: ${pattern.summary}`
60      };
61    }
62    
63    // Fall back to full processing if not similar enough
64    return this.processEmail(email);
65  }
66
67  private isSimilarEnough(email: Email, pattern: TriageResult): boolean {
68    // Implement similarity logic
69    const sameDomain = email.from.split('@')[1] === pattern.emailId.split('@')[1];
70    const similarSubject = this.calculateSimilarity(email.subject, pattern.summary) > 0.7;
71    
72    return sameDomain && similarSubject;
73  }
74
75  private calculateSimilarity(text1: string, text2: string): number {
76    // Simple similarity calculation
77    const words1 = text1.toLowerCase().split(/\s+/);
78    const words2 = text2.toLowerCase().split(/\s+/);
79    const intersection = words1.filter(word => words2.includes(word));
80    
81    return intersection.length / Math.max(words1.length, words2.length);
82  }
83}

Future Enhancements

Machine Learning Integration

Train custom models on your email data:

typescript

1class CustomModelTrainer {
2  async trainOrganizationModel(historicalData: Email[]): Promise<any> {
3    // Prepare training data
4    const trainingExamples = historicalData.map(email => ({
5      input: `${email.subject}\n${email.body}`,
6      output: email.category, // Assuming emails are pre-labeled
7      features: this.extractFeatures(email)
8    }));
9
10    // Fine-tune base model with organization-specific data
11    const customModel = await this.fineTuneModel({
12      baseModel: 'gpt-4o-mini',
13      trainingData: trainingExamples,
14      hyperparameters: {
15        learningRate: 0.0001,
16        epochs: 3,
17        batchSize: 16
18      }
19    });
20
21    return customModel;
22  }
23
24  private extractFeatures(email: Email) {
25    return {
26      senderDomain: email.from.split('@')[1],
27      hasAttachments: email.body.includes('attach'),
28      wordCount: email.body.split(/\s+/).length,
29      timeOfDay: email.timestamp.getHours(),
30      dayOfWeek: email.timestamp.getDay(),
31      containsNumbers: /\d/.test(email.subject + email.body),
32      urgencyIndicators: this.countUrgencyWords(email.subject + email.body)
33    };
34  }
35
36  private countUrgencyWords(text: string): number {
37    const urgencyWords = ['urgent', 'asap', 'immediate', 'emergency', 'critical'];
38    return urgencyWords.reduce((count, word) => 
39      count + (text.toLowerCase().match(new RegExp(word, 'g')) || []).length, 0
40    );
41  }
42
43  private async fineTuneModel(config: any): Promise<any> {
44    // Implementation would depend on your ML platform
45    console.log('Training custom model with config:', config);
46    // Return mock model for example
47    return { modelId: 'custom-email-classifier-v1', version: '1.0' };
48  }
49}

Workflow Automation

Trigger automated actions based on triage results:

typescript

1class WorkflowEngine {
2  private workflows: Map<string, WorkflowDefinition> = new Map();
3
4  constructor() {
5    this.setupDefaultWorkflows();
6  }
7
8  async executeWorkflow(result: TriageResult, email: Email): Promise<void> {
9    const workflowKey = `${result.category}-${result.priority}`;
10    const workflow = this.workflows.get(workflowKey) || this.workflows.get('default');
11    
12    if (workflow) {
13      await this.runWorkflow(workflow, result, email);
14    }
15  }
16
17  private setupDefaultWorkflows() {
18    // Urgent support workflow
19    this.workflows.set('support-urgent', {
20      steps: [
21        { action: 'createTicket', priority: 'urgent' },
22        { action: 'notifyOnCall', medium: 'sms' },
23        { action: 'escalateToManager', delay: 300000 } // 5 minutes
24      ]
25    });
26
27    // High-value sales lead workflow
28    this.workflows.set('sales-high', {
29      steps: [
30        { action: 'createLead', source: 'email' },
31        { action: 'assignToTopRep', criteria: 'revenue' },
32        { action: 'scheduleFollowUp', delay: 3600000 } // 1 hour
33      ]
34    });
35
36    // HR application workflow
37    this.workflows.set('hr-medium', {
38      steps: [
39        { action: 'parseResume', extractData: true },
40        { action: 'screenCandidate', automated: true },
41        { action: 'scheduleInterview', conditional: 'passed_screening' }
42      ]
43    });
44  }
45
46  private async runWorkflow(workflow: WorkflowDefinition, result: TriageResult, email: Email) {
47    for (const step of workflow.steps) {
48      try {
49        await this.executeStep(step, result, email);
50        
51        if (step.delay) {
52          await this.scheduleDelayedStep(step, result, email, step.delay);
53        }
54      } catch (error) {
55        console.error(`Workflow step failed:`, error);
56        // Continue with other steps or implement retry logic
57      }
58    }
59  }
60
61  private async executeStep(step: WorkflowStep, result: TriageResult, email: Email) {
62    switch (step.action) {
63      case 'createTicket':
64        await this.createSupportTicket(email, result, step.priority);
65        break;
66        
67      case 'notifyOnCall':
68        await this.notifyOnCallEngineer(result, step.medium);
69        break;
70        
71      case 'createLead':
72        await this.createCRMLead(email, result, step.source);
73        break;
74        
75      case 'assignToTopRep':
76        await this.assignToSalesRep(result, step.criteria);
77        break;
78        
79      case 'scheduleFollowUp':
80        await this.scheduleFollowUpCall(result);
81        break;
82        
83      default:
84        console.log(`Unknown workflow action: ${step.action}`);
85    }
86  }
87
88  private async scheduleDelayedStep(
89    step: WorkflowStep, 
90    result: TriageResult, 
91    email: Email, 
92    delay: number
93  ) {
94    setTimeout(async () => {
95      await this.executeStep(step, result, email);
96    }, delay);
97  }
98
99  // Workflow action implementations
100  private async createSupportTicket(email: Email, result: TriageResult, priority?: string) {
101    console.log(`Creating ${priority || result.priority} priority support ticket`);
102    // Implementation here
103  }
104
105  private async notifyOnCallEngineer(result: TriageResult, medium: string) {
106    console.log(`Notifying on-call engineer via ${medium}`);
107    // Implementation here
108  }
109
110  private async createCRMLead(email: Email, result: TriageResult, source: string) {
111    console.log(`Creating CRM lead from ${source}`);
112    // Implementation here
113  }
114
115  private async assignToSalesRep(result: TriageResult, criteria: string) {
116    console.log(`Assigning to sales rep based on ${criteria}`);
117    // Implementation here
118  }
119
120  private async scheduleFollowUpCall(result: TriageResult) {
121    console.log('Scheduling follow-up call');
122    // Implementation here
123  }
124}
125
126interface WorkflowDefinition {
127  steps: WorkflowStep[];
128}
129
130interface WorkflowStep {
131  action: string;
132  delay?: number;
133  priority?: string;
134  medium?: string;
135  source?: string;
136  criteria?: string;
137  extractData?: boolean;
138  automated?: boolean;
139  conditional?: string;
140}

Continuous Learning

Implement feedback loops:

typescript

1class ContinuousLearningSystem {
2  private feedbackStore: FeedbackData[] = [];
3  private retrainingThreshold = 100; // Retrain after 100 feedback entries
4
5  async recordFeedback(
6    emailId: string, 
7    prediction: TriageResult, 
8    actualCategory: string, 
9    userFeedback: string
10  ) {
11    const feedback: FeedbackData = {
12      emailId,
13      predictedCategory: prediction.category,
14      predictedAgent: prediction.assignedAgent,
15      actualCategory,
16      userFeedback,
17      timestamp: new Date(),
18      confidence: prediction.confidence
19    };
20
21    this.feedbackStore.push(feedback);
22    
23    // Analyze feedback for immediate improvements
24    await this.analyzeFeedback(feedback);
25    
26    // Check if retraining is needed
27    if (await this.shouldRetrain()) {
28      await this.triggerModelRetraining();
29    }
30  }
31
32  private async analyzeFeedback(feedback: FeedbackData) {
33    // Identify systematic errors
34    const recentFeedback = this.getRecentFeedback(7); // Last 7 days
35    const errorPatterns = this.identifyErrorPatterns(recentFeedback);
36    
37    if (errorPatterns.length > 0) {
38      console.log('Error patterns detected:', errorPatterns);
39      await this.adjustAgentThresholds(errorPatterns);
40    }
41  }
42
43  private getRecentFeedback(days: number): FeedbackData[] {
44    const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000);
45    return this.feedbackStore.filter(fb => fb.timestamp > cutoff);
46  }
47
48  private identifyErrorPatterns(feedback: FeedbackData[]): ErrorPattern[] {
49    const patterns: ErrorPattern[] = [];
50    
51    // Group by predicted vs actual category
52    const errorGroups = feedback
53      .filter(fb => fb.predictedCategory !== fb.actualCategory)
54      .reduce((groups, fb) => {
55        const key = `${fb.predictedCategory}->${fb.actualCategory}`;
56        if (!groups[key]) groups[key] = [];
57        groups[key].push(fb);
58        return groups;
59      }, {} as Record<string, FeedbackData[]>);
60
61    // Identify patterns with high frequency
62    Object.entries(errorGroups).forEach(([key, errors]) => {
63      if (errors.length >= 3) { // Pattern threshold
64        patterns.push({
65          type: 'misclassification',
66          from: key.split('->')[0],
67          to: key.split('->')[1],
68          frequency: errors.length,
69          avgConfidence: errors.reduce((sum, e) => sum + e.confidence, 0) / errors.length
70        });
71      }
72    });
73
74    return patterns;
75  }
76
77  private async adjustAgentThresholds(patterns: ErrorPattern[]) {
78    for (const pattern of patterns) {
79      if (pattern.type === 'misclassification' && pattern.avgConfidence > 80) {
80        // High confidence but wrong - adjust agent sensitivity
81        console.log(`Adjusting threshold for ${pattern.from} agent due to overconfidence`);
82        await this.updateAgentConfiguration(pattern.from, {
83          confidenceThreshold: pattern.avgConfidence + 5,
84          reviewRequired: true
85        });
86      }
87    }
88  }
89
90  private async shouldRetrain(): Promise<boolean> {
91    const totalFeedback = this.feedbackStore.length;
92    const errorRate = this.calculateErrorRate();
93    
94    return totalFeedback >= this.retrainingThreshold || errorRate > 0.15;
95  }
96
97  private calculateErrorRate(): number {
98    const recent = this.getRecentFeedback(30); // Last 30 days
99    if (recent.length === 0) return 0;
100    
101    const errors = recent.filter(fb => fb.predictedCategory !== fb.actualCategory);
102    return errors.length / recent.length;
103  }
104
105  private async triggerModelRetraining() {
106    console.log('Triggering model retraining with new feedback data');
107    
108    const trainingData = this.feedbackStore.map(fb => ({
109      email: fb.emailId,
110      correctCategory: fb.actualCategory,
111      feedback: fb.userFeedback
112    }));
113
114    // Trigger retraining job
115    await this.scheduleRetrainingJob(trainingData);
116    
117    // Clear processed feedback
118    this.feedbackStore = [];
119  }
120
121  private async scheduleRetrainingJob(trainingData: any[]) {
122    // Implementation would depend on your ML infrastructure
123    console.log(`Scheduling retraining job with ${trainingData.length} examples`);
124    
125    // Example: Submit to ML training pipeline
126    // await mlPipeline.submitTrainingJob({
127    //   data: trainingData,
128    //   model: 'email-triage-v2',
129    //   priority: 'normal'
130    // });
131  }
132
133  private async updateAgentConfiguration(agentName: string, updates: any) {
134    console.log(`Updating ${agentName} configuration:`, updates);
135    // Update agent configuration in database or config store
136  }
137}
138
139interface FeedbackData {
140  emailId: string;
141  predictedCategory: string;
142  predictedAgent: string;
143  actualCategory: string;
144  userFeedback: string;
145  timestamp: Date;
146  confidence: number;
147}
148
149interface ErrorPattern {
150  type: string;
151  from: string;
152  to: string;
153  frequency: number;
154  avgConfidence: number;
155}

Conclusion

Multi-agent systems provide a powerful framework for building sophisticated AI applications. By breaking complex problems into specialized components, we achieve better accuracy, maintainability, and scalability than single-agent approaches.

The email triage system demonstrates key multi-agent concepts:

Specialist agents with focused expertise
Supervisor coordination for decision-making
Parallel processing for efficiency
Conflict resolution when multiple agents compete

Key takeaways from building this system:

Specialization beats generalization - focused agents consistently outperform general-purpose ones
Parallel evaluation dramatically improves response times
Robust error handling ensures system reliability
Continuous learning keeps the system improving over time
Cost optimization through caching and tiered models makes production deployment viable

This architecture scales well to other domains and can be extended with additional agents, custom routing logic, and external integrations. Whether you're building systems for content moderation, financial analysis, healthcare triage, or legal document review, the multi-agent approach provides a solid foundation for complex AI applications.

The repository of this guide can be found here on GitHub.

From Email Triage to Incident Analysis: The Power of Specialized AI

While building custom multi-agent systems like this email triage example provides valuable learning and flexibility, many organizations struggle with another critical challenge: incident detection and analysis.

Uptime Agent specializes in solving the incident analysis problem with AI. Instead of chasing incidents manually, Uptime Agent automatically catches and analyzes them for you, providing intelligent insights that help teams respond faster and more effectively.

The multi-agent architecture principles we've explored in this email triage system - having specialized agents work together to solve complex problems - apply directly to incident management. Just as our email triage system routes communications to the right specialists, effective incident response requires intelligent routing, analysis, and coordination.

Whether you're building multi-agent systems for email triage, content moderation, or other automation tasks, the core concept remains the same: specialized AI agents working together deliver better results than any single general-purpose solution.

Ready to see how AI can transform your incident response? Discover how Uptime Agent can help you stop chasing incidents and catch them automatically.

This example demonstrates the fundamental concepts of multi-agent systems using LangChain.js and LangGraph. The complete code is available in the GitHub repository accompanying this blog post.