cadence
Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
Top Related Projects
AWS SDK for the Go programming language.
The official AWS SDK for Java 1.x. The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
The official AWS SDK for Ruby.
Quick Overview
Cadence is an open-source, scalable, and resilient distributed system for orchestrating background tasks and workflows. It is designed to handle long-running, complex business processes that require high availability and fault tolerance.
Pros
- Scalability: Cadence is designed to handle high-volume, high-throughput workloads, making it suitable for large-scale applications.
- Fault Tolerance: Cadence provides built-in fault tolerance, ensuring that workflows can continue even in the face of failures or outages.
- Flexibility: Cadence supports a wide range of workflow patterns, including long-running, parallel, and event-driven workflows.
- Language Agnostic: Cadence provides client libraries for various programming languages, allowing developers to use the language of their choice.
Cons
- Complexity: Cadence has a relatively steep learning curve, as it introduces a new set of concepts and abstractions for workflow management.
- Operational Overhead: Running a Cadence cluster requires additional infrastructure and operational overhead, which may not be suitable for small-scale projects.
- Limited Ecosystem: Compared to some other workflow management systems, Cadence has a smaller ecosystem of tools and integrations, which may limit its adoption in certain environments.
- Performance Overhead: Depending on the complexity of the workflows, Cadence may introduce some performance overhead, which should be considered when designing high-performance applications.
Code Examples
Here are a few code examples demonstrating the usage of Cadence:
Defining a Workflow
package main
import (
"context"
"time"
"github.com/uber-go/cadence-client/activity"
"github.com/uber-go/cadence-client/workflow"
)
func SampleWorkflow(ctx workflow.Context) error {
ao := workflow.ActivityOptions{
ScheduleToStartTimeout: time.Minute,
StartToCloseTimeout: time.Minute,
HeartbeatTimeout: time.Second * 20,
}
ctx = workflow.WithActivityOptions(ctx, ao)
err := workflow.ExecuteActivity(ctx, SampleActivity, "Hello").Get(ctx, nil)
if err != nil {
return err
}
return nil
}
func SampleActivity(ctx context.Context, msg string) error {
activity.GetLogger(ctx).Info("Received message: ", msg)
return nil
}
This example demonstrates how to define a simple workflow and an activity in Cadence.
Executing a Workflow
package main
import (
"log"
"github.com/uber-go/cadence-client/client"
"github.com/uber-go/cadence-client/worker"
)
func main() {
// Create a Cadence client
c, err := client.NewClient(client.Options{
HostPort: "cadence.example.com:7933",
})
if err != nil {
log.Fatalln("Unable to create Cadence client:", err)
}
// Execute the workflow
we, err := c.ExecuteWorkflow(context.Background(), client.StartWorkflowOptions{
TaskQueue: "sample-task-queue",
}, "SampleWorkflow", nil)
if err != nil {
log.Fatalln("Unable to execute workflow:", err)
}
log.Println("Workflow started:", we.GetID(), we.GetRunID())
}
This example shows how to execute a workflow using the Cadence client.
Registering a Worker
package main
import (
"log"
"github.com/uber-go/cadence-client/worker"
)
func main() {
// Create a Cadence worker
w := worker.New(
"cadence.example.com:7933",
"sample-task-queue",
worker.Options{
MaxConcurrentActivityExecutionSize: 10,
MaxConcurrentDecisionTaskExecutionSize: 10,
},
)
// Register the workflow and activity implementations
w.RegisterWork
Competitor Comparisons
AWS SDK for the Go programming language.
Pros of aws/aws-sdk-go
- Comprehensive coverage of AWS services: The aws/aws-sdk-go repository provides a wide range of AWS service integrations, making it a powerful tool for building applications that leverage the AWS ecosystem.
- Active development and community support: The project has a large and active community, with frequent updates and improvements to the SDK.
- Detailed documentation and examples: The project's documentation is extensive, providing clear guidance and examples for using the SDK.
Cons of aws/aws-sdk-go
- Complexity: The breadth of AWS services covered by the SDK can make it complex to use, especially for developers new to the AWS platform.
- Performance overhead: The SDK can add some performance overhead to applications, particularly for simple operations that don't require the full functionality of the SDK.
Code Comparison
Here's a brief code comparison between aws/aws-sdk-go and uber/cadence:
aws/aws-sdk-go (creating an S3 client):
sess := session.Must(session.NewSession())
s3Client := s3.New(sess)
uber/cadence (creating a Cadence client):
cadenceClient, err := client.NewClient(
client.Options{
HostPort: "cadence.example.com:7933",
IdentityToken: "your-identity-token",
MetricsHandler: metrics.Noop,
Logger: logger,
},
)
The official AWS SDK for Java 1.x. The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
Pros of aws/aws-sdk-java
- Comprehensive coverage of AWS services, allowing developers to interact with a wide range of cloud-based offerings.
- Actively maintained and updated, ensuring compatibility with the latest AWS features and services.
- Extensive documentation and community support, making it easier for developers to get started and troubleshoot issues.
Cons of aws/aws-sdk-java
- Complexity and size of the library, which can be overwhelming for developers working with a limited set of AWS services.
- Potential performance overhead due to the comprehensive nature of the library, which may not be necessary for all use cases.
- Dependency on the AWS ecosystem, which may not be suitable for developers working with other cloud providers or non-AWS services.
Code Comparison
Cadence (uber/cadence):
func (wh *WorkflowHandler) PollForDecisionTask(
ctx context.Context,
request *workflowservice.PollForDecisionTaskRequest,
) (*workflowservice.PollForDecisionTaskResponse, error) {
// ...
return wh.service.PollForDecisionTask(ctx, request)
}
AWS SDK for Java (aws/aws-sdk-java):
AmazonS3 s3 = AmazonS3ClientBuilder.standard()
.withRegion(Regions.US_WEST_2)
.build();
ListObjectsV2Request request = new ListObjectsV2Request()
.withBucketName("my-bucket")
.withMaxKeys(2);
ListObjectsV2Result result = s3.listObjectsV2(request);
The official AWS SDK for Ruby.
Pros of aws/aws-sdk-ruby
- Comprehensive coverage of AWS services, allowing developers to interact with a wide range of AWS offerings.
- Actively maintained and regularly updated to support the latest AWS features and services.
- Extensive documentation and community support, making it easier for developers to get started and troubleshoot issues.
Cons of aws/aws-sdk-ruby
- Complexity and verbosity of the API, which can make it challenging for beginners to get started.
- Potential performance overhead due to the comprehensive nature of the SDK, which may not be necessary for all use cases.
- Dependency on the AWS ecosystem, which may not be suitable for developers who need to work with non-AWS services.
Code Comparison
Cadence (uber/cadence):
func (wh *WorkflowHandler) StartWorkflow(ctx context.Context, request *workflowservice.StartWorkflowExecutionRequest) (*workflowservice.StartWorkflowExecutionResponse, error) {
wh.metricsClient.IncCounter(metrics.FrontendStartWorkflowExecutionScope, metrics.FrontendRequests)
wh.startWG.Wait()
if wh.isStopped() {
return nil, errShuttingDown
}
resp, err := wh.service.StartWorkflowExecution(ctx, request)
if err != nil {
wh.metricsClient.IncCounter(metrics.FrontendStartWorkflowExecutionScope, metrics.FrontendFailures)
}
return resp, err
}
AWS SDK for Ruby (aws/aws-sdk-ruby):
s3 = Aws::S3::Client.new(region: 'us-west-2')
resp = s3.list_buckets
resp.buckets.each do |bucket|
puts bucket.name
end
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Cadence
This repo contains the source code of the Cadence server and other tooling including CLI, schema tools, bench and canary.
You can implement your workflows with one of our client libraries. The Go and Java libraries are officially maintained by the Cadence team, while the Python and Ruby client libraries are developed by the community.
You can also use iWF as a DSL framework on top of Cadence.
See Maxim's talk at Data@Scale Conference for an architectural overview of Cadence.
Visit cadenceworkflow.io to learn more about Cadence. Join us in Cadence Documentation project. Feel free to raise an Issue or Pull Request there.
Community
- Github Discussion
- Best for Q&A, support/help, general discusion, and annoucement
- StackOverflow
- Best for Q&A and general discusion
- Github Issues
- Best for reporting bugs and feature requests
- Slack
- Best for contributing/development discussion
Getting Started
Start the cadence-server
To run Cadence services locally, we highly recommend that you use Cadence service docker to run the service. You can also follow the instructions to build and run it.
Please visit our documentation site for production/cluster setup.
Run the Samples
Try out the sample recipes for Go or Java to get started.
Use Cadence CLI
Cadence CLI can be used to operate workflows, tasklist, domain and even the clusters.
You can use the following ways to install Cadence CLI:
- Use brew to install CLI:
brew install cadence-workflow
- Follow the instructions if you need to install older versions of CLI via homebrew. Usually this is only needed when you are running a server of a too old version.
- Use docker image for CLI:
docker run --rm ubercadence/cli:<releaseVersion>
ordocker run --rm ubercadence/cli:master
. Be sure to update your image when you want to try new features:docker pull ubercadence/cli:master
- Build the CLI binary yourself, check out the repo and run
make cadence
to build all tools. See CONTRIBUTING for prerequisite of make command. - Build the CLI image yourself, see instructions
Cadence CLI is a powerful tool. The commands are organized by tabs. E.g. workflow
->batch
->start
, or admin
->workflow
->describe
.
Please read the documentation and always try out --help
on any tab to learn & explore.
Use Cadence Web
Try out Cadence Web UI to view your workflows on Cadence. (This is already available at localhost:8088 if you run Cadence with docker compose)
Contributing
We'd love your help in making Cadence great. Please review our contribution guide.
If you'd like to propose a new feature, first join the Slack channel to start a discussion and check if there are existing design discussions. Also peruse our design docs in case a feature has been designed but not yet implemented. Once you're sure the proposal is not covered elsewhere, please follow our proposal instructions.
Other binaries in this repo
Bench/stress test workflow tools
See bench documentation.
Periodical feature health check workflow tools(aka Canary)
See canary documentation.
Schema tools for SQL and Cassandra
The tools are for manual setup or upgrading database schema
- If server runs with Cassandra, Use Cadence Cassandra tool
- If server runs with SQL database, Use Cadence SQL tool
The easiest way to get the schema tool is via homebrew.
brew install cadence-workflow
also includes cadence-sql-tool
and cadence-cassandra-tool
.
- The schema files are located at
/usr/local/etc/cadence/schema/
. - To upgrade, make sure you remove the old ElasticSearch schema first:
mv /usr/local/etc/cadence/schema/elasticsearch /usr/local/etc/cadence/schema/elasticsearch.old && brew upgrade cadence-workflow
. Otherwise ElasticSearch schemas may not be able to get updated. - Follow the instructions if you need to install older versions of schema tools via homebrew.
However, easier way is to use new versions of schema tools with old versions of schemas.
All you need is to check out the older version of schemas from this repo. Run
git checkout v0.21.3
to get the v0.21.3 schemas in the schema folder.
Stargazers over time
License
MIT License, please see LICENSE for details.
Top Related Projects
AWS SDK for the Go programming language.
The official AWS SDK for Java 1.x. The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
The official AWS SDK for Ruby.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot