Convert Figma logo to code with AI

moosefs logomoosefs

MooseFS Distributed Storage – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System / Software-Defined Storage

1,679
207
1,679
166
View on GitHub

Top Related Projects

ceph's avatar

ceph

14,039

Ceph is a distributed object, block, and file storage platform

gluster's avatar

glusterfs

4,690

Gluster Filesystem : Build your distributed storage in minutes

minio's avatar

minio

47,355

MinIO is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license.

seaweedfs's avatar

seaweedfs

22,566

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

juicedata's avatar

juicefs

10,722

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

Quick Overview

MooseFS is an open-source, distributed file system designed for high availability, fault tolerance, and scalability. It provides a POSIX-compliant interface and allows for easy expansion of storage capacity by adding new servers to the cluster.

Pros

  • Highly scalable and easily expandable storage solution
  • Fault-tolerant with data replication and automatic recovery
  • POSIX-compliant, allowing for seamless integration with existing applications
  • Supports snapshots and trash bin for data recovery

Cons

  • Requires a dedicated metadata server, which can be a single point of failure
  • Limited support for Windows clients (primarily Linux-focused)
  • May have performance overhead compared to local file systems for certain workloads
  • Documentation can be sparse or outdated in some areas

Getting Started

To set up a basic MooseFS cluster:

  1. Install MooseFS on all nodes:

    sudo apt-get update
sudo apt-get install moosefs-master moosefs-chunkserver moosefs-client

  2. Configure the master server:

    sudo vim /etc/mfs/mfsmaster.cfg
# Set WORKING_USER and WORKING_GROUP

  3. Configure chunkservers:

    sudo vim /etc/mfs/mfschunkserver.cfg
# Set MASTER_HOST to the IP of the master server

  4. Start services:

    sudo systemctl start moosefs-master
sudo systemctl start moosefs-chunkserver

  5. Mount the filesystem on a client:

    sudo mkdir /mnt/moose
sudo mfsmount /mnt/moose -H master_ip

For more detailed setup instructions and advanced configurations, refer to the official MooseFS documentation.

Competitor Comparisons

ceph logo

ceph

14,039

Ceph is a distributed object, block, and file storage platform

Pros of Ceph

  • More feature-rich and versatile, supporting object, block, and file storage
  • Better scalability, capable of handling exabyte-scale deployments
  • Stronger consistency model, ensuring data integrity in distributed environments

Cons of Ceph

  • Higher complexity, requiring more resources and expertise to set up and maintain
  • Steeper learning curve for administrators and developers
  • Potentially higher latency for certain operations due to its distributed nature

Code Comparison

MooseFS client mount example:

mfsmount /mnt/moose -H mfsmaster

Ceph client mount example:

sudo mount -t ceph {ip-address}:6789:/ /mnt/cephfs -o name=admin,secret=AQC...

Both systems use different approaches for mounting, with Ceph requiring more configuration parameters due to its more complex architecture.

MooseFS focuses on simplicity and ease of use, while Ceph prioritizes scalability and versatility. MooseFS may be more suitable for smaller deployments or users seeking a straightforward distributed file system. Ceph, on the other hand, is better suited for large-scale, multi-purpose storage solutions in enterprise environments.

gluster logo

glusterfs

4,690

Gluster Filesystem : Build your distributed storage in minutes

Pros of GlusterFS

  • More mature and widely adopted in enterprise environments
  • Supports a wider range of storage configurations and use cases
  • Better integration with popular cloud platforms and container orchestration systems

Cons of GlusterFS

  • Can be more complex to set up and manage, especially for smaller deployments
  • May have higher resource overhead for certain workloads
  • Performance can be inconsistent in some scenarios, particularly with small files

Code Comparison

MooseFS client mount example:

mfsmount /mnt/moose -H mfsmaster

GlusterFS client mount example:

mount -t glusterfs server:/volume /mnt/gluster

Both systems use FUSE (Filesystem in Userspace) for client-side mounting, but GlusterFS offers more flexibility in terms of volume types and configurations.

MooseFS focuses on simplicity and ease of use, while GlusterFS provides more advanced features and customization options. The choice between the two often depends on specific use cases, scalability requirements, and existing infrastructure.

minio logo

minio

47,355

MinIO is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license.

Pros of MinIO

  • Designed specifically for object storage, offering S3 compatibility out of the box
  • Lightweight and easy to deploy, with a single binary for multiple platforms
  • Supports distributed mode for high availability and scalability

Cons of MinIO

  • Limited support for traditional file system operations compared to MooseFS
  • May require additional configuration for advanced features like erasure coding

Code Comparison

MinIO (Go):

mc := minio.New(endpoint, accessKeyID, secretAccessKey, useSSL)
_, err := mc.PutObject(bucketName, objectName, reader, objectSize, minio.PutObjectOptions{ContentType: "application/octet-stream"})

MooseFS (C):

mfs_init();
mfs_openfile(filename, O_WRONLY | O_CREAT);
mfs_write(fd, buffer, length);
mfs_close(fd);

Key Differences

  • MinIO focuses on object storage with S3 compatibility, while MooseFS is a distributed file system
  • MinIO offers a simpler deployment process, whereas MooseFS provides more traditional file system features
  • MinIO's API is more cloud-native, while MooseFS follows a POSIX-like interface

Use Cases

  • MinIO: Cloud-native applications, S3-compatible storage needs
  • MooseFS: Traditional file system requirements, large-scale distributed storage
seaweedfs logo

seaweedfs

22,566

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

Pros of SeaweedFS

  • Simpler architecture and easier to set up
  • Better performance for small files and high concurrency scenarios
  • Built-in support for object storage and S3 API compatibility

Cons of SeaweedFS

  • Less mature and battle-tested compared to MooseFS
  • Fewer advanced features like snapshots and quotas
  • Limited support for POSIX file system semantics

Code Comparison

SeaweedFS (Go):

func (vs *VolumeServer) autoVacuum() {
    for {
        if vs.isStopping {
            return
        }
        vs.vacuum()
        time.Sleep(time.Duration(vs.vacuumPeriodMinutes) * time.Minute)
    }
}

MooseFS (C):

int fs_read(void *buff, uint64_t size, uint64_t offset, uint8_t flags) {
    uint32_t ssize;
    uint8_t *ptr;
    const uint8_t *rptr;
    uint32_t rsize;
    int ret;
    // ... (implementation details)
}

Both projects implement distributed file systems, but SeaweedFS focuses on simplicity and object storage, while MooseFS offers a more traditional POSIX-compliant file system. SeaweedFS is written in Go, making it easier to deploy and maintain, while MooseFS is implemented in C for potentially better performance in certain scenarios.

juicedata logo

juicefs

10,722

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

Pros of JuiceFS

  • Cloud-native design with support for various object storage backends
  • POSIX-compliant with strong consistency and atomic operations
  • Built-in data compression and encryption features

Cons of JuiceFS

  • Relatively newer project with a smaller community compared to MooseFS
  • May have higher latency for certain operations due to its distributed nature

Code Comparison

MooseFS:

int mfs_setattr(mfs_context_t *ctx, const char *path, struct stat *stbuf, int to_set) {
    int status = mfs_node_setattr(ctx, path, stbuf, to_set);
    return (status == MFS_STATUS_OK) ? 0 : -EIO;
}

JuiceFS:

func (v *VFS) SetAttr(ctx context.Context, ino Ino, set uint32, attr *Attr) syscall.Errno {
    err := v.Meta.SetAttr(ctx, ino, set, attr)
    return errno(err)
}

Both systems implement similar functionality for setting file attributes, but JuiceFS uses Go and has a more modern, context-aware API design.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

MooseFS logo

MooseFS – A Petabyte Distributed File System

MooseFS is a Petabyte Open Source Network Distributed File System. It is easy to deploy and maintain, highly reliable, fault tolerant, highly performing, easily scalable and POSIX compliant.

MooseFS spreads data over a number of commodity servers, which are visible to the user as one resource. For standard file operations MooseFS acts like ordinary Unix-like file system:

  • A hierarchical structure – directory tree
  • Stores POSIX file attributes – permissions, last access and modification times, etc.
  • Supports ACLs
  • Supports POSIX and BSD file locks – including support for distributed file locking
  • Supports special files – block and character devices, pipes and sockets
  • Supports symbolic links – file names pointing to target files, not necessarily on MooseFS
  • Supports hard links – different names of files which refer to the same data on MooseFS

Distinctive MooseFS features:

  • High reliability – files are stored in several copies on separate servers. The number of copies is a configurable parameter, even per each file
  • No Single Point of Failure – all hardware and software components may be redundant
  • Parallel data operations – many clients can access many files concurrently
  • Capacity can be dynamically expanded by simply adding new servers/disks on the fly
  • Retired hardware may be removed on the fly
  • Deleted files are retained for a configurable period of time (a file system level "trash bin")
  • Coherent, "atomic" snapshots of files, even while the files are being written/accessed
  • Access to the file system can be limited based on IP address and/or password (similarly as in NFS)
  • Data tiering – supports different storage policies for different files/directories in Storage Classes mechanism
  • Per-directory, "project" quotas – configurable per RAW space, usable space, number of inodes with hard and soft quotas support
  • Apart from file system storage, MooseFS also provides block storage (mfsbdev)
  • Efficient, pure C implementation
  • Ethernet support

Supported platforms

MooseFS can be installed on any POSIX compliant operating system including various Linux distributions, FreeBSD and macOS:

  • Ubuntu
  • Debian
  • RHEL / CentOS
  • OpenSUSE
  • FreeBSD
  • macOS

MooseFS Client uses FUSE library, available for: Linux & BSD and macOS.

There is a separate MooseFS Client for Microsoft Windows available, built on top of Dokany.

Getting started

You can install MooseFS using your favourite package manager on one of the following platforms using officially supported repositories:

  • Ubuntu 16 / 18 / 20 / 22 / 24
  • Debian 9 / 10 / 11 / 12 / 13
  • RHEL / CentOS 7 / 8 / 9
  • FreeBSD 11 / 12 / 13 / 14
  • macOS 10.12+
  • Ubuntu 20 / 22 – Raspberry Pi
  • Debian 11 / 12 – Raspberry Pi

Packages for CentOS 6 are also available, but no longer supported.

Debian packages are compatible with Proxmox. CentOS packages are compatible with Rocky Linux, AlmaLinux and openSUSE Leap.

Minimal set of packages, which are needed to run MooseFS:

  • moosefs-master MooseFS Master Server for metadata servers,
  • moosefs-chunkserver MooseFS Chunkserver for data storage servers,
  • moosefs-client MooseFS Client – client side package to mount the filesystem.

Source code

Feel free to download the source code from our GitHub code repository!

Install the following dependencies before building MooseFS from sources:

  • Debian/Ubuntu: sudo apt install build-essential libpcap-dev zlib1g-dev libfuse3-dev pkg-config (if you don't have FUSE v. 3 in your system, use sudo apt install build-essential libpcap-dev zlib1g-dev libfuse-dev pkg-config)
  • CentOS/RHEL: sudo yum install gcc make libpcap-devel zlib-devel fuse3-devel pkgconfig (if you don't have FUSE v. 3 in your system, use sudo yum install gcc make libpcap-devel zlib-devel fuse-devel pkgconfig)

Recommended packages:

  • Debian/Ubuntu: sudo apt install fuse3 (if you don't have FUSE v. 3 in your system, use sudo apt install fuse)
  • CentOS/RHEL: sudo yum install fuse3 (if you don't have FUSE v. 3 in your system, use sudo yum install fuse)

Building MooseFS on Linux can be easily done by running ./linux_build.sh. Similarly, use ./freebsd_build.sh in order to build MooseFS on FreeBSD and respectively ./macosx_build.sh on macOS. Remember that these scripts do not install binaries (i.e. do not run make install) at the end. Run this command manually.

Minimal setup

Just three steps to have MooseFS up and running:

1. Install at least one Master Server

  1. Install moosefs-master package
  2. Prepare default config (as root):
cd /etc/mfs
cp mfsmaster.cfg.sample mfsmaster.cfg
cp mfsexports.cfg.sample mfsexports.cfg
  1. Prepare the metadata file (as root):
cd /var/lib/mfs
cp metadata.mfs.empty metadata.mfs
chown mfs:mfs metadata.mfs
rm metadata.mfs.empty
  1. Run Master Server (as root): mfsmaster start
  2. Make this machine visible under mfsmaster name, e.g. by adding a DNS entry (recommended) or by adding it in /etc/hosts on all servers that run any of MooseFS components.

2. Install at least two Chunkservers

  1. Install moosefs-chunkserver package
  2. Prepare default config (as root):
cd /etc/mfs
cp mfschunkserver.cfg.sample mfschunkserver.cfg
cp mfshdd.cfg.sample mfshdd.cfg
  1. At the end of mfshdd.cfg file make one or more entries containing paths to HDDs / partitions designated for storing chunks, e.g.:
/mnt/chunks1
/mnt/chunks2
/mnt/chunks3

It is recommended to use XFS as an underlying filesystem for disks designated to store chunks. More than two Chunkservers are strongly recommended.

  1. Change the ownership and permissions to mfs:mfs to above mentioned locations:
chown mfs:mfs /mnt/chunks1 /mnt/chunks2 /mnt/chunks3
chmod 770 /mnt/chunks1 /mnt/chunks2 /mnt/chunks3
  1. Start the Chunkserver: mfschunkserver start

Repeat steps above for second (third, ...) Chunkserver.

3. Client side: mount MooseFS filesystem

  1. Install moosefs-client package
  2. Mount MooseFS (as root):
mkdir /mnt/mfs
mount -t moosefs mfsmaster: /mnt/mfs

or: mfsmount -H mfsmaster /mnt/mfs if the above method is not supported by your system

  1. You can also add an /etc/fstab entry to mount MooseFS during the system boot:
mfsmaster:    /mnt/mfs    moosefs    defaults,mfsdelayedinit    0 0

There are more configuration parameters available but most of them may stay with defaults. We do our best to keep MooseFS easy to deploy and maintain.

MooseFS, for testing purposes, can even be installed on a single machine!

Additional tools

Setting up moosefs-cli or moosefs-cgi with moosefs-cgiserv is also recommended – it gives you a possibility to monitor the cluster online:

  1. Install moosefs-cli moosefs-cgi moosefs-cgiserv packages (they are typically set up on the Master Server)
  2. Run MooseFS CGI Server (as root): mfscgiserv start
  3. Open http://mfsmaster:9425 in your web browser

It is also strongly recommended to set up at least one Metalogger on a different machine than Master Server (e.g. on one of Chunkservers). Metalogger constantly synchronizes and backups the metadata:

  1. Install moosefs-metalogger package
  2. Prepare default config (as root):
cd /etc/mfs
cp mfsmetalogger.cfg.sample mfsmetalogger.cfg
  1. Run Metalogger (as root): mfsmetalogger start

Refer to installation guides for more details.

Some facts

Contact us

Copyright

Copyright (c) 2008-2024 Jakub Kruszona-Zawadzki, Saglabs SA

This file is part of MooseFS.

MooseFS is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 2 (only).

MooseFS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with MooseFS; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02111-1301, USA or visit http://www.gnu.org/licenses/gpl-2.0.html.