Hatchet

What is Hatchet?

Hatchet is a robust platform for managing background tasks at scale, built on an open-source, fault-tolerant queue system. It enables developers to distribute workloads efficiently across workers, ensuring no tasks are dropped even under heavy load or application failures. Hatchet supports complex workflows through task orchestration, concurrency control, scheduling, and real-time observability, making it ideal for mission-critical applications requiring reliable and scalable background processing.

Key Features

• Durable Task Queue

  Ensures tasks are reliably ingested and delivered to workers at a manageable rate, with automatic retries and failure alerts to prevent task loss.

• Advanced Task Orchestration

  Supports complex workflows with directed acyclic graphs (DAGs), allowing tasks to be chained, parallelized, and orchestrated with durable execution.

• Flow Control and Rate Limiting

  Provides concurrency limits, rate limiting, and fairness policies (FIFO, LIFO, Round Robin, Priority Queues) to protect system stability and handle traffic spikes.

• Comprehensive Scheduling

  Includes cron jobs, one-time scheduling, and durable sleep capabilities to run tasks on precise schedules or delays.

• Built-in Observability and Alerting

  Real-time dashboards, logging integration, and Slack/email alerting enable proactive monitoring and quick issue resolution.

• Flexible Task Routing

  Supports sticky assignments and worker affinity to route tasks to the most appropriate worker instances for optimized processing.

Use Cases

• AI Agent Workflows : Define and orchestrate agentic workflows with automatic retries and parallelized actions to build resilient AI systems.
• Handling Traffic Spikes : Smooth out large spikes in user requests by queuing and rate-limiting tasks to maintain system responsiveness and fairness.
• Batch Processing : Process large datasets such as document indexing or image processing with durable task execution and mid-job failure recovery.
• Multi-Modal Workflow Orchestration : Coordinate complex workflows involving multiple input and output types with DAG-style execution and durable state management.
• Event-Driven Processing : Respond to internal or external events with reliable task execution and automatic event replay for correctness.

FAQs