Use the queue library you know
Run Celery, BullMQ, Sidekiq, a Laravel queue worker or your own consumer process. Configure the command and broker connection your application requires.
Process files, send messages and handle queued jobs without holding up a web response. Run a dedicated worker with its own start command, environment and resources.
Your code. Infrastructure included.
$ celery -A app worker
connected to configured queue
Ready to process the next job_
BACKGROUND WORKERS ON OPENSTEAD
A worker can use the same repository as your web application, with a different command and resource allocation. Your task library stays in charge of the queue.
Run Celery, BullMQ, Sidekiq, a Laravel queue worker or your own consumer process. Configure the command and broker connection your application requires.
Assign worker memory and CPU independently of the web app. Adjust application concurrency to fit the service’s capacity and the type of jobs it processes.
Use private connections to Key Value and databases in the same environment. Inspect runtime output from the worker’s service page.
MADE FOR YOUR STACK
Deploy your application repository twice: once to receive requests, and once to process work. Each service can have its own command, variables and instance plan.
Repository your-team / application
Service type Background worker
Start command
php artisan queue:work redis
--sleep=3 --tries=3 --timeout=90
Environment
QUEUE_CONNECTION=redis
REDIS_HOST=your-key-value-private-hostUse your service credentials and tune timeouts for the jobs you run. Design jobs to tolerate retries and process restarts.
PUT IT TO WORK
Queue transactional messages and let a worker send them through your chosen delivery provider.
Handle data imports, transformations and uploaded files outside the HTTP request lifecycle.
Generate reports or synchronize data with external systems in a separate application process.
HOW IT WORKS
Choose the application repository or image and set the worker start command. Add the environment variables and credentials it needs.
Provision Key Value or use another compatible broker. Give the web application and worker the same queue connection settings.
Launch the worker, submit a test job and inspect its logs. Configure retries and concurrency in your task library.
Yes. Create a separate worker service from the same repository, then set its worker command and environment variables. The web app and worker have independent deployments and resources.
A worker has no public HTTP endpoint. Use a web service for incoming web traffic and a private service for an internal HTTP API.
The worker plan runs your consumer process. Provision Key Value or another compatible broker separately, and configure both the producer and consumer to use it.
Your task library controls retries, failed-job storage and acknowledgements. Use idempotent handlers where appropriate so repeating a task does not repeat an unintended side effect.
MAKE YOUR NEXT MOVE
Your code. Infrastructure included.