refactor(worker): Decouple capabilities from AI providers #226

New issue

Closed

opened 2026-01-24 17:49:51 +00:00 by jack · 0 comments

jack commented

2026-01-24 17:49:51 +00:00

Owner

Summary

Worker capabilities are currently tightly coupled to specific AI providers (e.g., summarize:mistral, observation:openai). Capabilities should describe what a worker can do, while the provider should be a separate, configurable property.

Current Problem

// Current: Provider embedded in capability name ❌
export type WorkerCapability =
  | 'observation:mistral'
  | 'observation:gemini'
  | 'observation:openrouter'
  | 'observation:sdk'
  | 'observation:openai'
  | 'summarize:mistral'
  | 'summarize:gemini'
  // ... explosion of combinations

Problems with this approach:

Combinatorial explosion: Every new provider requires new capability entries for each task type
Tight coupling: Can't easily switch providers without changing capabilities
Inflexible: Can't assign different providers to different workers for the same task type
Confusing semantics: Capability should describe the task, not the implementation

Proposed Solution

1. Abstract Capabilities

Capabilities describe what the worker can do:

export type WorkerCapability =
  // Core task types
  | 'observation'      // Generate observations from messages
  | 'summarize'        // Summarize sessions
  | 'embedding'        // Generate embeddings
  | 'qdrant-sync'      // Sync to vector database
  | 'context-generate' // Generate context
  | 'claudemd-generate'; // Generate CLAUDE.md content

2. Provider Types (grouped by function)

Providers are grouped by their function, not by capability:

// AI Providers
type LLMProvider = 'mistral' | 'gemini' | 'openrouter' | 'openai' | 'anthropic';
type EmbeddingProvider = 'local' | 'openai' | 'voyage';

// Infrastructure Providers
type VectorDBProvider = 'qdrant-local' | 'qdrant-cloud' | 'qdrant-remote';

interface ProviderConfig {
  // AI Providers
  llm?: LLMProvider;             // Used by: observation, summarize, claudemd-generate, context-generate
  embedding?: EmbeddingProvider; // Used by: embedding
  
  // Infrastructure Providers
  vectordb?: VectorDBProvider;   // Used by: qdrant-sync
}

3. Worker Configuration with Multiple Capabilities

IMPORTANT: Workers can have multiple capabilities. Each capability uses the appropriate provider based on its type:

interface WorkerConfig {
  capabilities: WorkerCapability[];  // Array - multiple capabilities per worker!
  providers: ProviderConfig;
}

Example: Multi-capability worker

{
  "capabilities": ["observation", "summarize", "embedding", "qdrant-sync", "claudemd-generate"],
  "providers": {
    "llm": "mistral",
    "embedding": "voyage",
    "vectordb": "qdrant-cloud"
  }
}

This worker can handle 5 different task types:

Capability	Provider Type	Provider Value
`observation`	`llm`	mistral
`summarize`	`llm`	mistral
`claudemd-generate`	`llm`	mistral
`embedding`	`embedding`	voyage
`qdrant-sync`	`vectordb`	qdrant-cloud

4. Capability-to-Provider Mapping

const CAPABILITY_PROVIDER_MAP: Record<WorkerCapability, keyof ProviderConfig | null> = {
  // AI-based capabilities
  'observation': 'llm',
  'summarize': 'llm',
  'claudemd-generate': 'llm',
  'context-generate': 'llm',
  'embedding': 'embedding',
  
  // Infrastructure capabilities
  'qdrant-sync': 'vectordb',
};

5. VectorDB Provider Configuration

The vectordb provider determines where embeddings are synced:

Provider	Description	Use Case
`qdrant-local`	Qdrant running locally	Development, single-machine setups
`qdrant-cloud`	Qdrant Cloud (managed)	Production, SaaS
`qdrant-remote`	Self-hosted remote Qdrant	Production, self-managed

Connection settings per provider:

{
  "VECTORDB_CONNECTIONS": {
    "qdrant-local": {
      "url": "http://localhost:6333"
    },
    "qdrant-cloud": {
      "url": "https://xxx.qdrant.io",
      "apiKey": "${QDRANT_CLOUD_API_KEY}"
    },
    "qdrant-remote": {
      "url": "https://qdrant.myserver.com",
      "apiKey": "${QDRANT_API_KEY}"
    }
  }
}

6. Settings-based Configuration Examples

Example 1: Specialized workers (one capability each)

{
  "WORKER_CONFIGS": [
    {
      "capabilities": ["observation", "summarize"],
      "providers": { "llm": "mistral" }
    },
    {
      "capabilities": ["embedding"],
      "providers": { "embedding": "voyage" }
    },
    {
      "capabilities": ["qdrant-sync"],
      "providers": { "vectordb": "qdrant-cloud" }
    }
  ]
}

Example 2: General-purpose workers (multiple capabilities)

{
  "WORKER_CONFIGS": [
    {
      "capabilities": ["observation", "summarize", "embedding", "claudemd-generate", "qdrant-sync"],
      "providers": {
        "llm": "openrouter",
        "embedding": "local",
        "vectordb": "qdrant-local"
      }
    },
    {
      "capabilities": ["observation", "summarize", "embedding", "claudemd-generate", "qdrant-sync"],
      "providers": {
        "llm": "gemini",
        "embedding": "openai",
        "vectordb": "qdrant-cloud"
      }
    }
  ]
}

Example 3: Mixed setup (specialized + general)

{
  "WORKER_CONFIGS": [
    {
      "comment": "Fast worker for quick tasks",
      "capabilities": ["observation", "summarize"],
      "providers": { "llm": "gemini" }
    },
    {
      "comment": "Dedicated embedding + sync worker",
      "capabilities": ["embedding", "qdrant-sync"],
      "providers": { 
        "embedding": "voyage",
        "vectordb": "qdrant-cloud"
      }
    },
    {
      "comment": "General fallback worker (local infrastructure)",
      "capabilities": ["observation", "summarize", "embedding", "claudemd-generate", "qdrant-sync"],
      "providers": {
        "llm": "openrouter",
        "embedding": "local",
        "vectordb": "qdrant-local"
      }
    }
  ]
}

7. Default Provider Fallback

If no provider is specified, use global default:

{
  "DEFAULT_PROVIDERS": {
    "llm": "openrouter",
    "embedding": "local",
    "vectordb": "qdrant-local"
  }
}

Benefits

Aspect	Before	After
Adding new provider	Add N new capabilities	Add 1 provider option
Switching providers	Change capability names	Change config
Mixed providers	Not possible	Assign per worker
Multi-capability workers	Complex capability lists	Simple capability array + provider config
VectorDB flexibility	Hardcoded	Configurable per worker
Type safety	Loose string unions	Strict enums
Readability	`summarize:mistral`	`summarize` + provider config

Migration Path

Phase 1: Add new abstract capabilities alongside existing ones
Phase 2: Update worker spawning to use new config format
Phase 3: Update task dispatcher to match abstract capabilities
Phase 4: Deprecate provider-specific capabilities
Phase 5: Remove deprecated capabilities

Files to Modify

packages/types/src/capabilities.ts - New capability types, provider types, mapping
packages/types/src/settings.ts - Worker config types, VectorDB connection config
packages/backend/src/services/worker-process-manager.ts - Spawn with provider config
packages/worker/src/ - Accept provider from config, use mapping
packages/backend/src/websocket/task-dispatcher.ts - Match abstract capabilities

#224 - Configurable worker capabilities and limits (depends on this)

Acceptance Criteria

Capabilities are abstract (no provider suffix)
Workers can have multiple capabilities with appropriate providers
Capability-to-provider mapping is clear and documented
AI providers (llm, embedding) are configurable per worker
VectorDB provider is configurable per worker
VectorDB connection settings are configurable
Default provider fallback works
Backwards compatible during migration
Documentation updated

## Summary Worker capabilities are currently tightly coupled to specific AI providers (e.g., `summarize:mistral`, `observation:openai`). Capabilities should describe **what** a worker can do, while the provider should be a separate, configurable property. ## Current Problem ```typescript // Current: Provider embedded in capability name ❌ export type WorkerCapability = | 'observation:mistral' | 'observation:gemini' | 'observation:openrouter' | 'observation:sdk' | 'observation:openai' | 'summarize:mistral' | 'summarize:gemini' // ... explosion of combinations ``` **Problems with this approach:** 1. **Combinatorial explosion**: Every new provider requires new capability entries for each task type 2. **Tight coupling**: Can't easily switch providers without changing capabilities 3. **Inflexible**: Can't assign different providers to different workers for the same task type 4. **Confusing semantics**: Capability should describe the task, not the implementation ## Proposed Solution ### 1. Abstract Capabilities Capabilities describe **what** the worker can do: ```typescript export type WorkerCapability = // Core task types | 'observation' // Generate observations from messages | 'summarize' // Summarize sessions | 'embedding' // Generate embeddings | 'qdrant-sync' // Sync to vector database | 'context-generate' // Generate context | 'claudemd-generate'; // Generate CLAUDE.md content ``` ### 2. Provider Types (grouped by function) Providers are grouped by their function, not by capability: ```typescript // AI Providers type LLMProvider = 'mistral' | 'gemini' | 'openrouter' | 'openai' | 'anthropic'; type EmbeddingProvider = 'local' | 'openai' | 'voyage'; // Infrastructure Providers type VectorDBProvider = 'qdrant-local' | 'qdrant-cloud' | 'qdrant-remote'; interface ProviderConfig { // AI Providers llm?: LLMProvider; // Used by: observation, summarize, claudemd-generate, context-generate embedding?: EmbeddingProvider; // Used by: embedding // Infrastructure Providers vectordb?: VectorDBProvider; // Used by: qdrant-sync } ``` ### 3. Worker Configuration with Multiple Capabilities **IMPORTANT:** Workers can have multiple capabilities. Each capability uses the appropriate provider based on its type: ```typescript interface WorkerConfig { capabilities: WorkerCapability[]; // Array - multiple capabilities per worker! providers: ProviderConfig; } ``` **Example: Multi-capability worker** ```json { "capabilities": ["observation", "summarize", "embedding", "qdrant-sync", "claudemd-generate"], "providers": { "llm": "mistral", "embedding": "voyage", "vectordb": "qdrant-cloud" } } ``` This worker can handle 5 different task types: | Capability | Provider Type | Provider Value | |------------|---------------|----------------| | `observation` | `llm` | mistral | | `summarize` | `llm` | mistral | | `claudemd-generate` | `llm` | mistral | | `embedding` | `embedding` | voyage | | `qdrant-sync` | `vectordb` | qdrant-cloud | ### 4. Capability-to-Provider Mapping ```typescript const CAPABILITY_PROVIDER_MAP: Record<WorkerCapability, keyof ProviderConfig | null> = { // AI-based capabilities 'observation': 'llm', 'summarize': 'llm', 'claudemd-generate': 'llm', 'context-generate': 'llm', 'embedding': 'embedding', // Infrastructure capabilities 'qdrant-sync': 'vectordb', }; ``` ### 5. VectorDB Provider Configuration The `vectordb` provider determines where embeddings are synced: | Provider | Description | Use Case | |----------|-------------|----------| | `qdrant-local` | Qdrant running locally | Development, single-machine setups | | `qdrant-cloud` | Qdrant Cloud (managed) | Production, SaaS | | `qdrant-remote` | Self-hosted remote Qdrant | Production, self-managed | **Connection settings per provider:** ```json { "VECTORDB_CONNECTIONS": { "qdrant-local": { "url": "http://localhost:6333" }, "qdrant-cloud": { "url": "https://xxx.qdrant.io", "apiKey": "${QDRANT_CLOUD_API_KEY}" }, "qdrant-remote": { "url": "https://qdrant.myserver.com", "apiKey": "${QDRANT_API_KEY}" } } } ``` ### 6. Settings-based Configuration Examples **Example 1: Specialized workers (one capability each)** ```json { "WORKER_CONFIGS": [ { "capabilities": ["observation", "summarize"], "providers": { "llm": "mistral" } }, { "capabilities": ["embedding"], "providers": { "embedding": "voyage" } }, { "capabilities": ["qdrant-sync"], "providers": { "vectordb": "qdrant-cloud" } } ] } ``` **Example 2: General-purpose workers (multiple capabilities)** ```json { "WORKER_CONFIGS": [ { "capabilities": ["observation", "summarize", "embedding", "claudemd-generate", "qdrant-sync"], "providers": { "llm": "openrouter", "embedding": "local", "vectordb": "qdrant-local" } }, { "capabilities": ["observation", "summarize", "embedding", "claudemd-generate", "qdrant-sync"], "providers": { "llm": "gemini", "embedding": "openai", "vectordb": "qdrant-cloud" } } ] } ``` **Example 3: Mixed setup (specialized + general)** ```json { "WORKER_CONFIGS": [ { "comment": "Fast worker for quick tasks", "capabilities": ["observation", "summarize"], "providers": { "llm": "gemini" } }, { "comment": "Dedicated embedding + sync worker", "capabilities": ["embedding", "qdrant-sync"], "providers": { "embedding": "voyage", "vectordb": "qdrant-cloud" } }, { "comment": "General fallback worker (local infrastructure)", "capabilities": ["observation", "summarize", "embedding", "claudemd-generate", "qdrant-sync"], "providers": { "llm": "openrouter", "embedding": "local", "vectordb": "qdrant-local" } } ] } ``` ### 7. Default Provider Fallback If no provider is specified, use global default: ```json { "DEFAULT_PROVIDERS": { "llm": "openrouter", "embedding": "local", "vectordb": "qdrant-local" } } ``` ## Benefits | Aspect | Before | After | |--------|--------|-------| | Adding new provider | Add N new capabilities | Add 1 provider option | | Switching providers | Change capability names | Change config | | Mixed providers | Not possible | Assign per worker | | Multi-capability workers | Complex capability lists | Simple capability array + provider config | | VectorDB flexibility | Hardcoded | Configurable per worker | | Type safety | Loose string unions | Strict enums | | Readability | `summarize:mistral` | `summarize` + provider config | ## Migration Path 1. **Phase 1**: Add new abstract capabilities alongside existing ones 2. **Phase 2**: Update worker spawning to use new config format 3. **Phase 3**: Update task dispatcher to match abstract capabilities 4. **Phase 4**: Deprecate provider-specific capabilities 5. **Phase 5**: Remove deprecated capabilities ## Files to Modify - `packages/types/src/capabilities.ts` - New capability types, provider types, mapping - `packages/types/src/settings.ts` - Worker config types, VectorDB connection config - `packages/backend/src/services/worker-process-manager.ts` - Spawn with provider config - `packages/worker/src/` - Accept provider from config, use mapping - `packages/backend/src/websocket/task-dispatcher.ts` - Match abstract capabilities ## Related Issues - #224 - Configurable worker capabilities and limits (depends on this) ## Acceptance Criteria - [ ] Capabilities are abstract (no provider suffix) - [ ] Workers can have multiple capabilities with appropriate providers - [ ] Capability-to-provider mapping is clear and documented - [ ] AI providers (llm, embedding) are configurable per worker - [ ] VectorDB provider is configurable per worker - [ ] VectorDB connection settings are configurable - [ ] Default provider fallback works - [ ] Backwards compatible during migration - [ ] Documentation updated