​
Overview

Middleware in Agenite allows you to intercept and modify agent behavior at various stages of execution. Middleware can be used for logging, error handling, state management, and more.

​
Interface

type AsyncGeneratorMiddleware<
  Yield extends MiddlewareBaseYieldValue = MiddlewareBaseYieldValue,
  Return = unknown,
  Next extends MiddlewareBaseNextValue = MiddlewareBaseNextValue,
  Generator extends AsyncGenerator<
    BaseYieldValue & {
      executionContext: StepContext<any>;
    },
    unknown,
    BaseNextValue
  > = BaseAgeniteIterateGenerator,
> = (
  generator: Generator,
  context: StepContext<any>
) => AsyncGenerator<
  Yield | GeneratorYieldType<Generator>,
  Return,
  Next | GeneratorNextType<Generator>
>;
The middleware wraps the agent’s execution generator, allowing you to:
  • Intercept and modify values yielded by the generator
  • Process the final return value
  • Handle next values passed into the generator
  • Access the execution context

​
Middleware Types

export type MiddlewareBaseYieldValue = {
  type: `middleware.${string}`;
  [key: string]: unknown;
};

export type MiddlewareBaseNextValue = {
  type: `middleware.${string}`;
  [key: string]: unknown;
};

​
Using middleware with an agent

const agent = new Agent({
  name: 'my-agent',
  provider: new BedrockProvider({ model: 'anthropic.claude-3-5-sonnet-20240620-v1:0' }),
  middlewares: [
    // Your middleware functions
    loggingMiddleware(),
    errorHandlingMiddleware()
  ],
  // ... other options
});

​
Custom middleware examples

​
Logging middleware

function loggingMiddleware(): AsyncGeneratorMiddleware {
  return async function* (generator, context) {
    console.log('Agent execution started');
    
    let nextValue = undefined;
    try {
      while (true) {
        const { value, done } = await generator.next(nextValue);
        
        if (done) {
          console.log('Agent execution completed');
          return value;
        }
        
        // Log each yielded value
        console.log('Event:', value.type);
        
        // Pass through to the caller
        nextValue = yield value;
      }
    } catch (error) {
      console.error('Agent execution error:', error);
      throw error;
    }
  };
}

​
Metrics middleware

function metricsMiddleware(): AsyncGeneratorMiddleware {
  return async function* (generator, context) {
    const metrics = {
      startTime: Date.now(),
      yields: 0,
      toolCalls: 0,
    };
    
    let nextValue = undefined;
    try {
      while (true) {
        const { value, done } = await generator.next(nextValue);
        
        if (done) {
          const duration = Date.now() - metrics.startTime;
          console.log('Execution metrics:', {
            ...metrics,
            duration,
          });
          return value;
        }
        
        metrics.yields++;
        if (value.type === 'agenite.tool-call.params') {
          metrics.toolCalls++;
        }
        
        nextValue = yield value;
      }
    } catch (error) {
      throw error;
    }
  };
}

​
State persistence middleware

function persistenceMiddleware(storage: Storage): AsyncGeneratorMiddleware {
  return async function* (generator, context) {
    // Load persisted state if exists
    const persistedState = await storage.get(`agent:${context.agent.agentConfig.name}`);
    if (persistedState) {
      Object.assign(context.state, JSON.parse(persistedState));
    }
    
    let nextValue = undefined;
    try {
      while (true) {
        const { value, done } = await generator.next(nextValue);
        
        if (done) {
          // Save final state
          await storage.set(
            `agent:${context.agent.agentConfig.name}`,
            JSON.stringify(value)
          );
          return value;
        }
        
        nextValue = yield value;
      }
    } catch (error) {
      throw error;
    }
  };
}

​
Best practices

  1. Order matters: Middlewares execute in the order specified in the array, with each wrapping the ones that come after it
  2. Keep it focused: Each middleware should have a single responsibility
  3. Error handling: Implement proper error handling within the middleware
  4. Performance: Be mindful of performance impacts, especially in middleware that runs for every yield
  5. Typing: Use proper TypeScript types for your custom middleware

​
Next steps