Dynein is Airbnb's Open-source Distributed Delayed Job Queueing System. Learn more about Dynein in this blog post: Dynein: Building an Open-source Distributed Delayed Job Queueing System.

Dynein's components are structured as Managed objects in Dropwizard. We have prepared an example application to demonstrate how to integrate Dynein in your service in ./example.

To run this example, follow the following steps:

1. Setup localstack. The following steps will be using the default ports of localstack.

2. Run this command to create the necessary SQS queues:

   aws sqs create-queue --endpoint-url http://localhost:4576 --queue-name example --region us-east-1
   aws sqs create-queue --endpoint-url http://localhost:4576 --queue-name inbound --region us-east-1

3. Run this command to create the DynamoDB table:

   aws dynamodb create-table --table-name dynein_schedules \
    --attribute-definitions AttributeName=shard_id,AttributeType=S AttributeName=date_token,AttributeType=S \
    --key-schema AttributeName=shard_id,KeyType=HASH AttributeName=date_token,KeyType=RANGE \
    --billing-mode PAY_PER_REQUEST --endpoint-url http://localhost:4569 --region us-east-1

4. cd example && gradle run --args='server example.yml'

5. In another terminal, use curl to schedule jobs. The following endpoints are available, which maps to scheduled, immediate, and sqs-delayed jobs:

   1. curl 'localhost:28080/schedule/scheduled'
   2. curl 'localhost:28080/schedule/immediate'
   3. curl 'localhost:28080/schedule/sqs-delayed'

   Each endpoint supports optional parameters delaySeconds and name. For example: curl 'localhost:28080/schedule/scheduled?name=testing-name&delaySeconds=15'

6. In the original terminal where the example is started, at the scheduled time, the following log lines will be printed:

   INFO [2019年12月10日 01:59:04,785] com.airbnb.dynein.example.ExampleJob: Running job with name: "testing-name"
   INFO [2019年12月10日 01:59:04,800] com.airbnb.dynein.example.worker.ManagedWorker: Successfully consumed job from queue
   

Here's an example from ./example/example.yml. We will use inline comments to explain what each option does.

scheduler:
 sqs:
 inboundQueueName: inbound # inboud queue name
 endpoint: "http://localhost:4576" # SQS endpoint. normally this line can be omitted, but it's useful for using localstack
 dynamoDb:
 schedulesTableName: dynein_schedules # dynamodb table name
 queryLimit: 10 # max number of results per query
 endpoint: "http://localhost:4569" # DynamoDB endpoint. normally this line can be omitted, but it's useful for localstack
 workers:
 numberOfWorkers: 1 # number of workers to start in this scheduler
 recoverStuckJobsLookAheadMs: 60000 # look ahead time for recovering stuck jobs
 threadPoolSize: 2 # must be at least 1 more than the number of workers
 partitionPolicy: STATIC # can also be `K8S` to use dynamic partitioning
 staticPartitionList: [0] # only needed for static partition policy
 heartbeat:
 monitorInterval: 60000 # the interval for heartbeat monitoring
 stallTolerance: 300000 # the minimum time for a worker to be considered stopped
 maxPartitions: 1 # max number of partitions. can only be increased without reassigning partitions for jobs already scheduled