rclone
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
Top Related Projects
Fast, secure, efficient backup program
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
Store securely encrypted backups in the cloud!
Deduplicating archiver with compression and authenticated encryption.
Open Source Continuous File Synchronization
Easily and securely send things from one computer to another :crocodile: :package:
Quick Overview
Rclone is an open-source command-line program designed for managing files on cloud storage. It supports a wide range of cloud storage providers and offers features like synchronization, encryption, and mounting cloud storage as a local disk. Rclone is often referred to as "rsync for cloud storage" due to its similar functionality and ease of use.
Pros
- Supports over 40 cloud storage providers, including popular ones like Google Drive, Dropbox, and Amazon S3
- Offers strong encryption and secure transfer options
- Provides a unified interface for managing files across different cloud storage services
- Highly customizable with numerous configuration options and filters
Cons
- Command-line interface may be intimidating for non-technical users
- Some advanced features require a deeper understanding of the tool and cloud storage concepts
- Performance can be slower compared to native sync clients for some providers
- Limited GUI options, although third-party GUIs are available
Code Examples
- Copying files from local to remote storage:
rclone copy /path/to/local/files remote:path/to/files
- Syncing files between two remote storage providers:
rclone sync remote1:path/to/files remote2:path/to/files
- Mounting a remote storage as a local disk:
rclone mount remote:path/to/files /path/to/local/mount
- Encrypting files before uploading to remote storage:
rclone copy --encrypt-remote "secret" /path/to/local/files remote:path/to/encrypted/files
Getting Started
-
Install rclone:
- On Linux/macOS:
curl https://rclone.org/install.sh | sudo bash
- On Windows: Download and run the installer from https://rclone.org/downloads/
- On Linux/macOS:
-
Configure a remote:
rclone config
Follow the prompts to set up your cloud storage provider.
-
Basic usage:
rclone ls remote:path # List files rclone copy source:path dest:path # Copy files rclone sync source:path dest:path # Sync files
For more detailed instructions and advanced usage, refer to the official documentation at https://rclone.org/docs/.
Competitor Comparisons
Fast, secure, efficient backup program
Pros of restic
- Focuses on backup and restore functionality with built-in deduplication and encryption
- Supports multiple storage backends, including local, SFTP, and cloud providers
- Offers snapshot-based backups with efficient incremental updates
Cons of restic
- Limited to backup and restore operations, lacking file synchronization features
- Requires separate management of repository initialization and maintenance
- May have slower performance for large datasets compared to rclone's sync operations
Code comparison
restic:
repo, err := restic.OpenRepository(repoPath)
if err != nil {
return err
}
defer repo.Close()
rclone:
fsrc, fdst := cmd.NewFsSrcDst(args)
err = sync.Sync(ctx, fdst, fsrc, opt)
if err != nil {
return err
}
The code snippets demonstrate the different focus of each tool. restic emphasizes repository management for backups, while rclone focuses on file synchronization between different storage locations.
Both projects are open-source and written in Go, offering cross-platform support. While restic specializes in secure, efficient backups with deduplication, rclone provides a more versatile toolkit for file operations across various cloud storage providers. The choice between them depends on specific use cases and requirements for data management and synchronization.
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
Pros of Kopia
- Built-in data deduplication and compression for efficient storage
- Supports content-defined chunking for better handling of large files
- Offers a user-friendly GUI in addition to CLI
Cons of Kopia
- Limited support for cloud storage providers compared to Rclone
- Lacks some advanced features like mounting remote filesystems
Code Comparison
Kopia:
session, err := repo.OpenSession(ctx, repo.WriteSessionOptions{
Purpose: "backup",
})
if err != nil {
log.Fatalf("Unable to open repository: %v", err)
}
Rclone:
fsrc, err := fs.NewFs(context.Background(), args[0])
if err != nil {
log.Fatalf("Failed to create source Fs: %v", err)
}
Both projects are written in Go and provide robust backup and sync functionality. Kopia focuses on efficient storage with built-in deduplication and compression, while Rclone offers broader support for cloud storage providers and more advanced features. Kopia's content-defined chunking can be advantageous for handling large files, but Rclone's extensive provider support and features like mounting remote filesystems give it an edge in versatility. The choice between the two depends on specific use cases and requirements.
Store securely encrypted backups in the cloud!
Pros of Duplicati
- User-friendly graphical interface for easier configuration and management
- Built-in encryption and compression features
- Supports creating full, incremental, and differential backups
Cons of Duplicati
- Slower performance for large file sets compared to Rclone
- Less flexible for advanced users who prefer command-line operations
- More resource-intensive due to its graphical interface and additional features
Code Comparison
Duplicati (C#):
using (var backend = DynamicLoader.BackendLoader.GetBackend(backendurl))
{
backend.Put(localfile, remotename);
}
Rclone (Go):
err := fs.CopyFile(ctx, dst, src)
if err != nil {
log.Fatalf("Failed to copy: %v", err)
}
Both examples demonstrate file transfer operations, but Rclone's implementation is more straightforward and lower-level, reflecting its focus on simplicity and performance. Duplicati's code shows its use of dynamic loading for backend support, highlighting its flexibility in supporting various storage providers.
Deduplicating archiver with compression and authenticated encryption.
Pros of Borg
- Efficient deduplication and compression, reducing storage space
- Built-in encryption for enhanced security
- Supports mounting backups as filesystems for easy browsing
Cons of Borg
- Limited to local and SSH-based backups
- Steeper learning curve compared to Rclone
- Less flexibility in terms of supported cloud storage providers
Code Comparison
Borg backup creation:
borg create /path/to/repo::archive_name /path/to/files
Rclone sync operation:
rclone sync /local/path remote:path
Both tools use command-line interfaces, but Borg focuses on creating and managing archives, while Rclone emphasizes syncing between different storage locations.
Borg is ideal for users prioritizing efficient local or SSH-based backups with strong encryption and deduplication. Rclone excels in versatility, supporting a wide range of cloud storage providers and offering more flexibility in file synchronization tasks.
The choice between Borg and Rclone depends on specific backup needs, storage preferences, and desired features. Borg is better suited for secure, space-efficient local backups, while Rclone is more appropriate for users requiring cloud storage integration and diverse synchronization options.
Open Source Continuous File Synchronization
Pros of Syncthing
- Continuous, real-time file synchronization
- Decentralized peer-to-peer architecture
- Built-in encryption and security features
Cons of Syncthing
- Limited support for cloud storage services
- Steeper learning curve for initial setup
- May consume more system resources for continuous syncing
Code Comparison
Syncthing (Go):
func (m *Model) Index(deviceID protocol.DeviceID, folder string, files []protocol.FileInfo, flags uint32, options []protocol.Option) {
m.fmut.Lock()
defer m.fmut.Unlock()
if !m.folderSharedWith(folder, deviceID) {
l.Infof("Unexpected index for folder %q from device %v", folder, deviceID)
return
}
}
Rclone (Go):
func (f *Fs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error) {
o := &Object{
fs: f,
remote: src.Remote(),
}
return o, o.Update(ctx, in, src, options...)
}
Both projects are written in Go and focus on file synchronization, but Rclone is primarily designed for cloud storage operations, while Syncthing emphasizes peer-to-peer file sharing and real-time synchronization.
Easily and securely send things from one computer to another :crocodile: :package:
Pros of croc
- Simpler and more user-friendly for quick file transfers between devices
- Built-in end-to-end encryption for secure file sharing
- No need for cloud storage or server setup
Cons of croc
- Limited functionality compared to rclone's extensive cloud storage support
- Less suitable for large-scale data synchronization or backup tasks
- Fewer configuration options and advanced features
Code comparison
croc:
func send(c *cli.Context) error {
// ... (code for sending files)
}
func receive(c *cli.Context) error {
// ... (code for receiving files)
}
rclone:
func (f *Fs) Copy(ctx context.Context, src Object, remote string) (Object, error) {
// ... (code for copying files between cloud storage providers)
}
func (f *Fs) Move(ctx context.Context, src Object, remote string) (Object, error) {
// ... (code for moving files between cloud storage providers)
}
The code snippets show that croc focuses on simple send and receive operations, while rclone provides more complex functionality for interacting with various cloud storage services.
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Website | Documentation | Download | Contributing | Changelog | Installation | Forum
Rclone
Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.
Storage providers
- 1Fichier :page_facing_up:
- Akamai Netstorage :page_facing_up:
- Alibaba Cloud (Aliyun) Object Storage System (OSS) :page_facing_up:
- Amazon S3 :page_facing_up:
- ArvanCloud Object Storage (AOS) :page_facing_up:
- Backblaze B2 :page_facing_up:
- Box :page_facing_up:
- Ceph :page_facing_up:
- China Mobile Ecloud Elastic Object Storage (EOS) :page_facing_up:
- Cloudflare R2 :page_facing_up:
- Citrix ShareFile :page_facing_up:
- DigitalOcean Spaces :page_facing_up:
- Digi Storage :page_facing_up:
- Dreamhost :page_facing_up:
- Dropbox :page_facing_up:
- Enterprise File Fabric :page_facing_up:
- Fastmail Files :page_facing_up:
- Files.com :page_facing_up:
- FTP :page_facing_up:
- GoFile :page_facing_up:
- Google Cloud Storage :page_facing_up:
- Google Drive :page_facing_up:
- Google Photos :page_facing_up:
- HDFS (Hadoop Distributed Filesystem) :page_facing_up:
- Hetzner Storage Box :page_facing_up:
- HiDrive :page_facing_up:
- HTTP :page_facing_up:
- Huawei Cloud Object Storage Service(OBS) :page_facing_up:
- iCloud Drive :page_facing_up:
- ImageKit :page_facing_up:
- Internet Archive :page_facing_up:
- Jottacloud :page_facing_up:
- IBM COS S3 :page_facing_up:
- IONOS Cloud :page_facing_up:
- Koofr :page_facing_up:
- Leviia Object Storage :page_facing_up:
- Liara Object Storage :page_facing_up:
- Linkbox :page_facing_up:
- Linode Object Storage :page_facing_up:
- Magalu Object Storage :page_facing_up:
- Mail.ru Cloud :page_facing_up:
- Memset Memstore :page_facing_up:
- Mega :page_facing_up:
- Memory :page_facing_up:
- Microsoft Azure Blob Storage :page_facing_up:
- Microsoft Azure Files Storage :page_facing_up:
- Microsoft OneDrive :page_facing_up:
- Minio :page_facing_up:
- Nextcloud :page_facing_up:
- OVH :page_facing_up:
- Blomp Cloud Storage :page_facing_up:
- OpenDrive :page_facing_up:
- OpenStack Swift :page_facing_up:
- Oracle Cloud Storage :page_facing_up:
- Oracle Object Storage :page_facing_up:
- Outscale :page_facing_up:
- ownCloud :page_facing_up:
- pCloud :page_facing_up:
- Petabox :page_facing_up:
- PikPak :page_facing_up:
- Pixeldrain :page_facing_up:
- premiumize.me :page_facing_up:
- put.io :page_facing_up:
- Proton Drive :page_facing_up:
- QingStor :page_facing_up:
- Qiniu Cloud Object Storage (Kodo) :page_facing_up:
- Quatrix :page_facing_up:
- Rackspace Cloud Files :page_facing_up:
- RackCorp Object Storage :page_facing_up:
- rsync.net :page_facing_up:
- Scaleway :page_facing_up:
- Seafile :page_facing_up:
- SeaweedFS :page_facing_up:
- Selectel Object Storage :page_facing_up:
- SFTP :page_facing_up:
- SMB / CIFS :page_facing_up:
- StackPath :page_facing_up:
- Storj :page_facing_up:
- SugarSync :page_facing_up:
- Synology C2 Object Storage :page_facing_up:
- Tencent Cloud Object Storage (COS) :page_facing_up:
- Uloz.to :page_facing_up:
- Wasabi :page_facing_up:
- WebDAV :page_facing_up:
- Yandex Disk :page_facing_up:
- Zoho WorkDrive :page_facing_up:
- The local filesystem :page_facing_up:
Please see the full list of all storage providers and their features
Virtual storage providers
These backends adapt or modify other storage providers
- Alias: rename existing remotes :page_facing_up:
- Cache: cache remotes (DEPRECATED) :page_facing_up:
- Chunker: split large files :page_facing_up:
- Combine: combine multiple remotes into a directory tree :page_facing_up:
- Compress: compress files :page_facing_up:
- Crypt: encrypt files :page_facing_up:
- Hasher: hash files :page_facing_up:
- Union: join multiple remotes to work together :page_facing_up:
Features
- MD5/SHA-1 hashes checked at all times for file integrity
- Timestamps preserved on files
- Partial syncs supported on a whole file basis
- Copy mode to just copy new/changed files
- Sync (one way) mode to make a directory identical
- Bisync (two way) to keep two directories in sync bidirectionally
- Check mode to check for file hash equality
- Can sync to and from network, e.g. two different cloud accounts
- Optional large file chunking (Chunker)
- Optional transparent compression (Compress)
- Optional encryption (Crypt)
- Optional FUSE mount (rclone mount)
- Multi-threaded downloads to local disk
- Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA
Installation & documentation
Please see the rclone website for:
Downloads
License
This is free software under the terms of the MIT license (check the COPYING file included in this package).
Top Related Projects
Fast, secure, efficient backup program
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.
Store securely encrypted backups in the cloud!
Deduplicating archiver with compression and authenticated encryption.
Open Source Continuous File Synchronization
Easily and securely send things from one computer to another :crocodile: :package:
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot