customable/claude-mem

Fork 0

feat: Capture all MCP tool usage (exclude only claude-mem's own MCP tools) #110

New issue

Closed

opened 2026-01-22 22:50:24 +00:00 by jack · 1 comment

jack commented

2026-01-22 22:50:24 +00:00

Owner

Problem

Claude-mem currently captures observations from native Claude Code tools (Read, Write, Bash, Grep, etc.) but may not capture tool usage from other MCP servers.

As users increasingly rely on MCP servers for specialized functionality (databases, APIs, browser automation, etc.), these interactions represent valuable context that should be preserved in memory.

Current Behavior

The PostToolUse hook likely only processes:

Native Claude Code tools (Read, Write, Edit, Bash, Grep, Glob, etc.)
Potentially ignoring tools from MCP servers like:
- mcp__forgejo__* (Forgejo/Gitea operations)
- mcp__playwright__* (Browser automation)
- mcp__context7__* (Documentation lookup)
- User-installed custom MCP servers

Desired Behavior

Capture all tool usage EXCEPT claude-mem's own MCP tools:

✅ Include:

All native Claude Code tools
All MCP tools from external servers (Forgejo, Playwright, Context7, etc.)
Custom user-installed MCP servers

❌ Exclude:

mcp__plugin_claude-mem_mcp-search__* (claude-mem's own search tools)
Internal claude-mem operations (would create circular observations)

Implementation Strategy

1. Update Tool Filtering Logic

Current skip list in save-hook.ts:

const SKIP_TOOLS = new Set([
  'ListMcpResourcesTool',
  'SlashCommand',
  'Skill',
  'TodoWrite',
  'AskUserQuestion'
]);

Enhanced filtering:

function shouldCaptureToolUse(toolName: string): boolean {
  // Skip meta-tools (low value)
  const META_TOOLS = new Set([
    'ListMcpResourcesTool',
    'SlashCommand',
    'Skill',
    'TodoWrite',
    'AskUserQuestion'
  ]);
  
  if (META_TOOLS.has(toolName)) {
    return false;
  }
  
  // Skip claude-mem's own MCP tools (avoid circular observations)
  if (toolName.startsWith('mcp__plugin_claude-mem_')) {
    return false;
  }
  
  // Capture everything else (native tools + external MCP tools)
  return true;
}

2. Handle MCP Tool Structure

MCP tools may have different input/output structures than native tools. Ensure the observation pipeline handles:

// Example Forgejo tool
{
  tool_name: "mcp__forgejo__create_issue",
  tool_input: {
    owner: "customable",
    repo: "claude-mem",
    title: "Issue title",
    body: "Issue body"
  },
  tool_response: {
    number: 123,
    url: "https://..."
  }
}

3. Observation Classification

MCP tools should be properly categorized:

function classifyToolType(toolName: string): 'native' | 'mcp' | 'custom' {
  if (toolName.startsWith('mcp__')) {
    return 'mcp';
  }
  // Native Claude Code tools
  const NATIVE_TOOLS = ['Read', 'Write', 'Edit', 'Bash', 'Grep', 'Glob', ...];
  if (NATIVE_TOOLS.includes(toolName)) {
    return 'native';
  }
  return 'custom';
}

This metadata could be stored in observations for future filtering/analysis.

Benefits

Complete session history - No gaps from MCP tool usage
Better context - Understand API calls, database queries, browser interactions
Cross-session learning - "How did I use the Playwright MCP last time?"
Debugging - Track what external tools were used and why
Token accounting - Accurately measure cost of MCP vs native tools

Example Use Cases

Forgejo MCP:

Observation: Created issue #109 in customable/claude-mem
- Tool: mcp__forgejo__create_issue
- Files referenced: /tmp/endless-mode-temp/src/hooks/save-hook.ts
- Token cost: ~500

Playwright MCP:

Observation: Navigated to login page and submitted credentials
- Tool: mcp__playwright__browser_navigate
- Followed by: mcp__playwright__browser_fill_form
- Success: Authentication succeeded

Context7 MCP:

Observation: Retrieved Next.js documentation for useEffect hook
- Tool: mcp__context7__query-docs
- Library: /vercel/next.js
- Relevant code snippets captured

Testing Strategy

Install test MCP servers (Forgejo, Playwright, Context7)
Execute various MCP tools
Verify observations are created
Confirm claude-mem's own MCP tools are excluded
Check observation quality (title, narrative, facts)

Consider impact on Endless Mode (#109) - MCP tools may have large responses
Token accounting for MCP usage
Privacy considerations for sensitive MCP operations (credentials, API keys)

Open Questions

Should we have per-MCP-server skip lists for low-value tools?
How to handle MCP tools with very large responses (e.g., browser screenshots)?
Should MCP tool observations have special formatting or metadata?
Do we need MCP-specific privacy tag handling?

## Problem Claude-mem currently captures observations from native Claude Code tools (Read, Write, Bash, Grep, etc.) but may not capture tool usage from other MCP servers. As users increasingly rely on MCP servers for specialized functionality (databases, APIs, browser automation, etc.), these interactions represent valuable context that should be preserved in memory. ## Current Behavior The PostToolUse hook likely only processes: - Native Claude Code tools (Read, Write, Edit, Bash, Grep, Glob, etc.) - Potentially ignoring tools from MCP servers like: - `mcp__forgejo__*` (Forgejo/Gitea operations) - `mcp__playwright__*` (Browser automation) - `mcp__context7__*` (Documentation lookup) - User-installed custom MCP servers ## Desired Behavior **Capture all tool usage EXCEPT claude-mem's own MCP tools:** ✅ **Include:** - All native Claude Code tools - All MCP tools from external servers (Forgejo, Playwright, Context7, etc.) - Custom user-installed MCP servers ❌ **Exclude:** - `mcp__plugin_claude-mem_mcp-search__*` (claude-mem's own search tools) - Internal claude-mem operations (would create circular observations) ## Implementation Strategy ### 1. Update Tool Filtering Logic Current skip list in `save-hook.ts`: ```typescript const SKIP_TOOLS = new Set([ 'ListMcpResourcesTool', 'SlashCommand', 'Skill', 'TodoWrite', 'AskUserQuestion' ]); ``` Enhanced filtering: ```typescript function shouldCaptureToolUse(toolName: string): boolean { // Skip meta-tools (low value) const META_TOOLS = new Set([ 'ListMcpResourcesTool', 'SlashCommand', 'Skill', 'TodoWrite', 'AskUserQuestion' ]); if (META_TOOLS.has(toolName)) { return false; } // Skip claude-mem's own MCP tools (avoid circular observations) if (toolName.startsWith('mcp__plugin_claude-mem_')) { return false; } // Capture everything else (native tools + external MCP tools) return true; } ``` ### 2. Handle MCP Tool Structure MCP tools may have different input/output structures than native tools. Ensure the observation pipeline handles: ```typescript // Example Forgejo tool { tool_name: "mcp__forgejo__create_issue", tool_input: { owner: "customable", repo: "claude-mem", title: "Issue title", body: "Issue body" }, tool_response: { number: 123, url: "https://..." } } ``` ### 3. Observation Classification MCP tools should be properly categorized: ```typescript function classifyToolType(toolName: string): 'native' | 'mcp' | 'custom' { if (toolName.startsWith('mcp__')) { return 'mcp'; } // Native Claude Code tools const NATIVE_TOOLS = ['Read', 'Write', 'Edit', 'Bash', 'Grep', 'Glob', ...]; if (NATIVE_TOOLS.includes(toolName)) { return 'native'; } return 'custom'; } ``` This metadata could be stored in observations for future filtering/analysis. ## Benefits 1. **Complete session history** - No gaps from MCP tool usage 2. **Better context** - Understand API calls, database queries, browser interactions 3. **Cross-session learning** - "How did I use the Playwright MCP last time?" 4. **Debugging** - Track what external tools were used and why 5. **Token accounting** - Accurately measure cost of MCP vs native tools ## Example Use Cases **Forgejo MCP:** ``` Observation: Created issue #109 in customable/claude-mem - Tool: mcp__forgejo__create_issue - Files referenced: /tmp/endless-mode-temp/src/hooks/save-hook.ts - Token cost: ~500 ``` **Playwright MCP:** ``` Observation: Navigated to login page and submitted credentials - Tool: mcp__playwright__browser_navigate - Followed by: mcp__playwright__browser_fill_form - Success: Authentication succeeded ``` **Context7 MCP:** ``` Observation: Retrieved Next.js documentation for useEffect hook - Tool: mcp__context7__query-docs - Library: /vercel/next.js - Relevant code snippets captured ``` ## Testing Strategy 1. Install test MCP servers (Forgejo, Playwright, Context7) 2. Execute various MCP tools 3. Verify observations are created 4. Confirm claude-mem's own MCP tools are excluded 5. Check observation quality (title, narrative, facts) ## Related Issues - Consider impact on Endless Mode (#109) - MCP tools may have large responses - Token accounting for MCP usage - Privacy considerations for sensitive MCP operations (credentials, API keys) ## Open Questions 1. Should we have per-MCP-server skip lists for low-value tools? 2. How to handle MCP tools with very large responses (e.g., browser screenshots)? 3. Should MCP tool observations have special formatting or metadata? 4. Do we need MCP-specific privacy tag handling?

jack added the

labels

2026-01-22 22:50:25 +00:00

jack referenced this issue

2026-01-22 22:54:07 +00:00

feat: Store MCP documentation lookups as searchable documents (Context7, etc.) #111

jonas.hanisch referenced this issue from a commit

2026-01-23 17:18:24 +00:00

feat(hooks): capture all MCP tool usage except claude-mem's own

jonas.hanisch closed this issue

2026-01-23 17:18:24 +00:00

jack commented

2026-01-23 17:18:32 +00:00

Author

Owner

Implemented in packages/hooks/src/handlers/post-tool-use.ts.

Changes:

Added shouldCaptureToolUse() function for cleaner filtering logic
Exclude mcp__plugin_claude-mem_* tools to avoid circular observations
Expanded IGNORED_TOOLS to skip meta-tools:
- AskUserQuestion
- ListMcpResourcesTool
- SlashCommand
- Skill

Result: All external MCP tools (Forgejo, Playwright, Context7, custom servers) are now captured as observations.

Implemented in `packages/hooks/src/handlers/post-tool-use.ts`. **Changes:** - Added `shouldCaptureToolUse()` function for cleaner filtering logic - Exclude `mcp__plugin_claude-mem_*` tools to avoid circular observations - Expanded `IGNORED_TOOLS` to skip meta-tools: - `AskUserQuestion` - `ListMcpResourcesTool` - `SlashCommand` - `Skill` **Result:** All external MCP tools (Forgejo, Playwright, Context7, custom servers) are now captured as observations.