fix: add stuck task recovery with orphan detection and retry #346

Merged

review.bot merged 1 commit from fix/340-stuck-tasks into develop

2026-03-02 14:00:50 +00:00

jack commented

2026-03-02 14:00:43 +00:00

Owner

Summary

checkTimeouts() now detects orphaned tasks (assigned to workers no longer connected) in addition to timed-out tasks
Stuck tasks are re-queued with exponential backoff instead of being permanently marked as timeout
Tasks only fail permanently after max retries (3) are exhausted
Stuck task check frequency increased from 5 min to 30 sec for faster recovery
Also fixes pre-existing TS errors in hooks-router.test.ts

Root causes addressed:

Tasks assigned to disconnected workers stayed in processing forever (no orphan detection)
Timed-out tasks were marked timeout but never retried
5-minute check interval meant up to 10 min before a stuck task was detected

Closes #340

## Summary - `checkTimeouts()` now detects **orphaned tasks** (assigned to workers no longer connected) in addition to timed-out tasks - Stuck tasks are **re-queued with exponential backoff** instead of being permanently marked as `timeout` - Tasks only fail permanently after max retries (3) are exhausted - Stuck task check frequency increased from **5 min to 30 sec** for faster recovery - Also fixes pre-existing TS errors in hooks-router.test.ts **Root causes addressed:** 1. Tasks assigned to disconnected workers stayed in `processing` forever (no orphan detection) 2. Timed-out tasks were marked `timeout` but never retried 3. 5-minute check interval meant up to 10 min before a stuck task was detected Closes #340

jack added 1 commit

2026-03-02 14:00:43 +00:00

fix: add stuck task recovery with orphan detection and retry (#340 ) b8c30213fc

- checkTimeouts() now detects orphaned tasks (assigned to disconnected
  workers) in addition to timed-out tasks
- Stuck tasks are re-queued with exponential backoff instead of being
  permanently marked as timeout
- Tasks only fail after max retries are exhausted
- Stuck task check frequency increased from 5min to 30sec
- Fix pre-existing TS errors in hooks-router.test.ts

review.bot

2026-03-02 14:00:44 +00:00

added the
component
api

priority
high

type
enhancement
labels
requested review from review.bot

review.bot reviewed

2026-03-02 14:00:48 +00:00

review.bot left a comment

Owner

AI Code Review (Devstral)

The changes look good and address the issue of stuck tasks by implementing orphan detection and retry logic. The test fixes are also appropriate.

Automated review by pr-reviewer

## AI Code Review (Devstral) The changes look good and address the issue of stuck tasks by implementing orphan detection and retry logic. The test fixes are also appropriate. --- *Automated review by [pr-reviewer](https://github.com/customable/pr-reviewer)*

review.bot added

and removed

labels

2026-03-02 14:00:48 +00:00