bug(worker): Task assignment ignores pendingTermination flag #223

New issue

Closed

opened 2026-01-24 17:46:22 +00:00 by jack · 0 comments

jack commented

2026-01-24 17:46:22 +00:00

Owner

Summary

Workers marked with pendingTermination = true can still receive new task assignments, defeating the purpose of graceful termination.

Current Behavior

When a worker is queued for termination via queueTermination(), the pendingTermination flag is set to true. However, findAvailableWorker() and findAvailableWorkerForAny() do not check this flag before returning a worker as available.

Result: A worker that completes its current task while pending termination can receive a new task instead of being terminated.

Affected Code

`packages/backend/src/websocket/worker-hub.ts`

Line 398-408 - findAvailableWorker():

findAvailableWorker(capability: WorkerCapability): ConnectedWorker | null {
  for (const worker of this.workers.values()) {
    if (
      !worker.currentTaskId &&
      worker.capabilities.includes(capability)
    ) {
      return worker;  // ❌ No check for pendingTermination
    }
  }
  return null;
}

Line 413-424 - findAvailableWorkerForAny():

findAvailableWorkerForAny(capabilities: WorkerCapability[]): ConnectedWorker | null {
  for (const worker of this.workers.values()) {
    if (!worker.currentTaskId) {  // ❌ No check for pendingTermination
      for (const cap of capabilities) {
        if (worker.capabilities.includes(cap)) {
          return worker;
        }
      }
    }
  }
  return null;
}

Expected Behavior

Workers with pendingTermination = true should be excluded from task assignment and should be terminated after completing their current task.

Fix

Add !worker.pendingTermination check to both methods:

// findAvailableWorker
if (
  !worker.currentTaskId &&
  !worker.pendingTermination &&  // ← ADD
  worker.capabilities.includes(capability)
) {
  return worker;
}

// findAvailableWorkerForAny
if (!worker.currentTaskId && !worker.pendingTermination) {  // ← ADD
  for (const cap of capabilities) {
    // ...
  }
}

Impact

Severity: Medium
Affected: Worker lifecycle management, graceful shutdown
Symptoms: Workers may not terminate when expected, resource accumulation

## Summary Workers marked with `pendingTermination = true` can still receive new task assignments, defeating the purpose of graceful termination. ## Current Behavior When a worker is queued for termination via `queueTermination()`, the `pendingTermination` flag is set to `true`. However, `findAvailableWorker()` and `findAvailableWorkerForAny()` do not check this flag before returning a worker as available. **Result:** A worker that completes its current task while pending termination can receive a new task instead of being terminated. ## Affected Code ### `packages/backend/src/websocket/worker-hub.ts` **Line 398-408** - `findAvailableWorker()`: ```typescript findAvailableWorker(capability: WorkerCapability): ConnectedWorker | null { for (const worker of this.workers.values()) { if ( !worker.currentTaskId && worker.capabilities.includes(capability) ) { return worker; // ❌ No check for pendingTermination } } return null; } ``` **Line 413-424** - `findAvailableWorkerForAny()`: ```typescript findAvailableWorkerForAny(capabilities: WorkerCapability[]): ConnectedWorker | null { for (const worker of this.workers.values()) { if (!worker.currentTaskId) { // ❌ No check for pendingTermination for (const cap of capabilities) { if (worker.capabilities.includes(cap)) { return worker; } } } } return null; } ``` ## Expected Behavior Workers with `pendingTermination = true` should be excluded from task assignment and should be terminated after completing their current task. ## Fix Add `!worker.pendingTermination` check to both methods: ```typescript // findAvailableWorker if ( !worker.currentTaskId && !worker.pendingTermination && // ← ADD worker.capabilities.includes(capability) ) { return worker; } // findAvailableWorkerForAny if (!worker.currentTaskId && !worker.pendingTermination) { // ← ADD for (const cap of capabilities) { // ... } } ``` ## Impact - **Severity:** Medium - **Affected:** Worker lifecycle management, graceful shutdown - **Symptoms:** Workers may not terminate when expected, resource accumulation