Overview

The dispatch endpoint allows you to run conversation completions in the background without keeping a connection open. This is particularly useful for:

• Long-running completions that may take several minutes
• Scenarios where the client may disconnect (page refresh, mobile apps)
• Batch processing where you want to fire-and-forget

How It Works

1. Dispatch: Call the dispatch endpoint with the same parameters as the complete endpoint. You'll receive a channelId in the response.

2. Subscribe: Use the channel subscribe endpoint to receive real-time events as the completion progresses.

3. Process: The completion runs in the background, publishing events to the channel including tokens, operations, and the final result.

Example Usage

// Step 1: Dispatch the completion
const dispatchResponse = await fetch('/api/v1/conversation/dispatch', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    backstory: "You are a helpful assistant",
    messages: [{ type: "user", text: "Analyze this complex data..." }]
  })
});

const { channelId } = await dispatchResponse.json();

// Step 2: Subscribe to the channel for updates
const subscribeResponse = await fetch(`/api/v1/channel/${channelId}/subscribe`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({})
});

// Step 3: Process streaming events
const reader = subscribeResponse.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const chunk = decoder.decode(value);
  const lines = chunk.split('\n').filter(line => line.trim());

  for (const line of lines) {
    const event = JSON.parse(line);
    console.log('Event:', event.type, event.data);

    if (event.type === 'message' && event.data.type === 'result') {
      console.log('Completion finished:', event.data);
    }
  }
}
javascript

Channel Events

The channel will receive the same events as the streaming complete endpoint:

• token: Individual tokens as they are generated
• message: Activity messages, function calls, etc.
• result: The final result
• error: Any errors that occurred

Important Considerations

• Channel Lifetime: Channels remain active during the completion. Once the completion finishes, the channel will no longer receive new events.

• Missed Events: If you subscribe after the completion has started, you may miss some events. Subscribe as soon as you receive the channel ID.

• Session Scope: The channel ID is scoped to your session for security.