Top Related Projects
SQL schema migration tool for Go.
A database migration tool. Supports SQL migrations and Go functions.
Database migrations. CLI and Golang library.
Flyway by Redgate • Database Migrations Made Easy.
Main Liquibase Source
🚀 A lightweight, framework-agnostic database migration tool.
Quick Overview
golang-migrate/migrate is a database migration tool and library for Go. It supports various database drivers and sources for migration files, allowing developers to manage database schema changes efficiently across different environments and database systems.
Pros
- Supports multiple database systems (e.g., PostgreSQL, MySQL, SQLite)
- Offers various migration sources (e.g., filesystem, GitHub, AWS S3)
- Provides both a CLI tool and a Go library for integration
- Allows for versioned migrations with up and down operations
Cons
- Learning curve for complex migration scenarios
- Limited support for some less common database systems
- Potential for conflicts when used in distributed environments
- Requires careful management of migration files to avoid issues
Code Examples
- Creating a new migration:
import (
"github.com/golang-migrate/migrate/v4"
_ "github.com/golang-migrate/migrate/v4/database/postgres"
_ "github.com/golang-migrate/migrate/v4/source/file"
)
func main() {
m, err := migrate.New(
"file://path/to/migrations",
"postgres://user:password@localhost:5432/dbname?sslmode=disable")
if err != nil {
log.Fatal(err)
}
if err := m.Up(); err != nil {
log.Fatal(err)
}
}
- Rolling back a migration:
if err := m.Down(); err != nil {
log.Fatal(err)
}
- Migrating to a specific version:
if err := m.Migrate(1234); err != nil {
log.Fatal(err)
}
Getting Started
-
Install the migrate CLI:
go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest
-
Create migration files:
migrate create -ext sql -dir migrations -seq init_schema
-
Run migrations:
migrate -path ./migrations -database "postgresql://user:password@localhost:5432/dbname?sslmode=disable" up
-
To use as a library, import the required packages:
import ( "github.com/golang-migrate/migrate/v4" _ "github.com/golang-migrate/migrate/v4/database/postgres" _ "github.com/golang-migrate/migrate/v4/source/file" )
Competitor Comparisons
SQL schema migration tool for Go.
Pros of sql-migrate
- Simpler setup and usage, especially for Go projects
- Built-in support for multiple database dialects
- Easier to manage migrations with Go code instead of raw SQL files
Cons of sql-migrate
- Less flexibility in migration file formats
- Fewer database drivers supported out-of-the-box
- Limited CLI functionality compared to migrate
Code Comparison
sql-migrate:
migrations := &migrate.FileMigrationSource{
Dir: "migrations",
}
n, err := migrate.Exec(db, "postgres", migrations, migrate.Up)
if err != nil {
// Handle error
}
migrate:
m, err := migrate.New("file://migrations", "postgres://localhost:5432/database")
if err != nil {
// Handle error
}
err = m.Up()
if err != nil {
// Handle error
}
Both libraries offer straightforward ways to run migrations, but sql-migrate's approach is more Go-centric, while migrate provides a more database-agnostic interface.
sql-migrate is better suited for smaller Go projects with simpler migration needs, while migrate offers more flexibility and features for complex, multi-database environments. The choice between the two depends on the specific requirements of your project and your preferred workflow.
A database migration tool. Supports SQL migrations and Go functions.
Pros of Goose
- Simpler and more straightforward API, making it easier to use for beginners
- Built-in support for Go's
database/sql
package, allowing for easier integration with existing Go projects - Supports both SQL and Go-based migrations, offering flexibility in migration implementation
Cons of Goose
- Less extensive database support compared to Migrate (e.g., no MongoDB or Cassandra)
- Fewer advanced features, such as migration locking or version control integration
- Limited CLI options compared to Migrate's more comprehensive command-line interface
Code Comparison
Goose migration example:
package main
import (
"database/sql"
)
func Up(tx *sql.Tx) error {
_, err := tx.Exec("CREATE TABLE users (id INT, name TEXT);")
return err
}
Migrate migration example:
package main
import (
"github.com/golang-migrate/migrate/v4"
_ "github.com/golang-migrate/migrate/v4/database/postgres"
_ "github.com/golang-migrate/migrate/v4/source/file"
)
func main() {
m, _ := migrate.New("file://migrations", "postgres://localhost:5432/database")
m.Up()
}
Both Goose and Migrate are popular database migration tools for Go projects. Goose offers simplicity and ease of use, while Migrate provides more extensive database support and advanced features. The choice between them depends on project requirements and complexity.
Database migrations. CLI and Golang library.
Pros of migrate
- Simpler and more lightweight, potentially easier for beginners
- Faster execution due to fewer features and dependencies
- More focused on core migration functionality
Cons of migrate
- Less actively maintained, with fewer recent updates
- Limited database support compared to golang-migrate/migrate
- Fewer advanced features and customization options
Code Comparison
migrate:
m, err := migrate.New("file://migrations", "postgres://localhost:5432/database")
if err != nil {
log.Fatal(err)
}
m.Steps(2)
golang-migrate/migrate:
m, err := migrate.New(
"file://migrations",
"postgres://localhost:5432/database?sslmode=enable"
)
if err != nil {
log.Fatal(err)
}
m.Steps(2)
The code structure is similar, but golang-migrate/migrate offers more configuration options and supports additional parameters in the database URL.
Both repositories provide Go-based database migration tools, but golang-migrate/migrate is more actively maintained and feature-rich. It supports a wider range of databases and offers more advanced functionality. However, migrate may be preferable for simpler projects or those prioritizing speed and simplicity over extensive features.
Flyway by Redgate • Database Migrations Made Easy.
Pros of Flyway
- Supports a wider range of databases, including Oracle, DB2, and SAP HANA
- Offers both a command-line tool and a Java API for integration into applications
- Provides a more comprehensive ecosystem with commercial support and additional features
Cons of Flyway
- Requires Java runtime, which may not be ideal for all environments
- Can be more complex to set up and configure compared to migrate
- Limited support for NoSQL databases
Code Comparison
migrate:
m, err := migrate.New(
"file://migrations",
"postgres://localhost:5432/database?sslmode=enable"
)
if err != nil {
log.Fatal(err)
}
m.Steps(2)
Flyway:
Flyway flyway = Flyway.configure().dataSource(url, user, password).load();
flyway.migrate();
Summary
Both migrate and Flyway are powerful database migration tools. migrate offers simplicity and Go language integration, while Flyway provides broader database support and a more extensive ecosystem. The choice between them depends on your specific project requirements, preferred programming language, and the databases you need to support.
Main Liquibase Source
Pros of Liquibase
- More extensive database support, including NoSQL databases
- Robust rollback capabilities and change tracking
- Supports multiple file formats for change logs (XML, YAML, JSON, SQL)
Cons of Liquibase
- Steeper learning curve due to more complex configuration
- Slower execution compared to Migrate for simple migrations
- Java-based, which may not integrate as seamlessly with non-Java projects
Code Comparison
Migrate:
m, err := migrate.New(
"file://migrations",
"postgres://localhost:5432/database?sslmode=enable"
)
if err != nil {
log.Fatal(err)
}
Liquibase:
<changeSet id="1" author="example">
<createTable tableName="users">
<column name="id" type="int">
<constraints primaryKey="true" nullable="false"/>
</column>
<column name="name" type="varchar(50)"/>
</createTable>
</changeSet>
Migrate focuses on simplicity and ease of use, making it ideal for Go projects and straightforward migrations. It supports fewer databases but offers faster execution for basic operations.
Liquibase provides a more comprehensive solution with advanced features like rollbacks and change tracking. It's better suited for complex database schemas and projects requiring detailed migration management across various database types.
Choose Migrate for Go projects with simple migration needs, and Liquibase for more complex, database-agnostic scenarios requiring advanced migration management.
🚀 A lightweight, framework-agnostic database migration tool.
Pros of dbmate
- Simpler setup and usage, with a focus on ease of use
- Supports schema dumps for easier version control and review
- Provides a web UI for managing migrations (dbmate-ui)
Cons of dbmate
- Supports fewer databases compared to migrate
- Less active development and smaller community
- Lacks some advanced features like migration locking
Code Comparison
migrate:
m, err := migrate.New("file://migrations", "postgres://localhost:5432/database")
if err != nil {
log.Fatal(err)
}
m.Steps(2)
dbmate:
dbmate new create_users_table
dbmate up
dbmate dump
Key Differences
- migrate is written in Go, while dbmate is written in Ruby
- migrate offers more granular control over migration steps
- dbmate focuses on simplicity and provides a CLI-first approach
- migrate has broader database support and more active development
- dbmate includes schema dump functionality out of the box
Both tools serve the purpose of database migration management, but they cater to different preferences and use cases. migrate is more suitable for complex scenarios and Go-centric projects, while dbmate excels in simplicity and ease of use, especially for Ruby developers or those who prefer a straightforward CLI tool.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
migrate
Database migrations written in Go. Use as CLI or import as library.
- Migrate reads migrations from sources and applies them in correct order to a database.
- Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
- Database drivers don't assume things or try to correct user input. When in doubt, fail.
Forked from mattes/migrate
Databases
Database drivers run migrations. Add a new database?
- PostgreSQL
- PGX v4
- PGX v5
- Redshift
- Ql
- Cassandra / ScyllaDB
- SQLite
- SQLite3 (todo #165)
- SQLCipher
- MySQL / MariaDB
- Neo4j
- MongoDB
- CrateDB (todo #170)
- Shell (todo #171)
- Google Cloud Spanner
- CockroachDB
- YugabyteDB
- ClickHouse
- Firebird
- MS SQL Server
- rqlite
Database URLs
Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?param1=true¶m2=false
Any reserved URL characters need to be escaped. Note, the %
character also needs to be escaped
Explicitly, the following characters need to be escaped:
!
, #
, $
, %
, &
, '
, (
, )
, *
, +
, ,
, /
, :
, ;
, =
, ?
, @
, [
, ]
It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:
$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$
Migration Sources
Source drivers read migrations from local or remote sources. Add a new source?
- Filesystem - read from filesystem
- io/fs - read from a Go io/fs
- Go-Bindata - read from embedded binary data (jteeuwen/go-bindata)
- pkger - read from embedded binary data (markbates/pkger)
- GitHub - read from remote GitHub repositories
- GitHub Enterprise - read from remote GitHub Enterprise repositories
- Bitbucket - read from remote Bitbucket repositories
- Gitlab - read from remote Gitlab repositories
- AWS S3 - read from Amazon Web Services S3
- Google Cloud Storage - read from Google Cloud Platform Storage
CLI usage
- Simple wrapper around this library.
- Handles ctrl+c (SIGINT) gracefully.
- No config search paths, no config files, no magic ENV var injections.
Basic usage
$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2
Docker usage
$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate
-path=/migrations/ -database postgres://localhost:5432/database up 2
Use in your Go project
- API is stable and frozen for this release (v3 & v4).
- Uses Go modules to manage dependencies.
- To help prevent database corruptions, it supports graceful stops via
GracefulStop chan bool
. - Bring your own logger.
- Uses
io.Reader
streams internally for low memory overhead. - Thread-safe and no goroutine leaks.
import (
"github.com/golang-migrate/migrate/v4"
_ "github.com/golang-migrate/migrate/v4/database/postgres"
_ "github.com/golang-migrate/migrate/v4/source/github"
)
func main() {
m, err := migrate.New(
"github://mattes:personal-access-token@mattes/migrate_test",
"postgres://localhost:5432/database?sslmode=enable")
m.Steps(2)
}
Want to use an existing database client?
import (
"database/sql"
_ "github.com/lib/pq"
"github.com/golang-migrate/migrate/v4"
"github.com/golang-migrate/migrate/v4/database/postgres"
_ "github.com/golang-migrate/migrate/v4/source/file"
)
func main() {
db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable")
driver, err := postgres.WithInstance(db, &postgres.Config{})
m, err := migrate.NewWithDatabaseInstance(
"file:///migrations",
"postgres", driver)
m.Up() // or m.Steps(2) if you want to explicitly set the number of migrations to run
}
Getting started
Go to getting started
Tutorials
(more tutorials to come)
Migration files
Each migration has an up and down migration. Why?
1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql
Best practices: How to write migrations.
Coming from another db migration tool?
Check out migradaptor. Note: migradaptor is not affiliated or supported by this project
Versions
Version | Supported? | Import | Notes |
---|---|---|---|
master | :white_check_mark: | import "github.com/golang-migrate/migrate/v4" | New features and bug fixes arrive here first |
v4 | :white_check_mark: | import "github.com/golang-migrate/migrate/v4" | Used for stable releases |
v3 | :x: | import "github.com/golang-migrate/migrate" (with package manager) or import "gopkg.in/golang-migrate/migrate.v3" (not recommended) | DO NOT USE - No longer supported |
Development and Contributing
Yes, please! Makefile
is your friend,
read the development guide.
Also have a look at the FAQ.
Looking for alternatives? https://awesome-go.com/#database.
Top Related Projects
SQL schema migration tool for Go.
A database migration tool. Supports SQL migrations and Go functions.
Database migrations. CLI and Golang library.
Flyway by Redgate • Database Migrations Made Easy.
Main Liquibase Source
🚀 A lightweight, framework-agnostic database migration tool.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot