vcluster
vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
Top Related Projects
Home for Cluster API, a subproject of sig-cluster-lifecycle
Little helper to run CNCF's k3s in Docker
Run Kubernetes locally
Lightweight Kubernetes
Kubernetes IN Docker - local clusters for testing Kubernetes
Backup and migrate Kubernetes applications and their persistent volumes
Quick Overview
vcluster is an open-source project that creates virtual Kubernetes clusters within a host Kubernetes cluster. It allows users to create lightweight, isolated environments for development, testing, and multi-tenancy scenarios without the overhead of spinning up separate physical clusters.
Pros
- Efficient resource utilization by running multiple virtual clusters on a single host cluster
- Improved isolation and security between different teams or projects
- Easy to set up and manage compared to full-fledged Kubernetes clusters
- Supports most standard Kubernetes tools and workflows
Cons
- Some limitations in terms of supported Kubernetes features compared to full clusters
- Potential performance overhead due to the virtualization layer
- Requires careful planning for resource allocation to avoid overloading the host cluster
- Learning curve for understanding the concept and management of virtual clusters
Getting Started
To get started with vcluster, follow these steps:
- Install the vcluster CLI:
curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-$(uname -s)-$(uname -m)" && chmod +x vcluster && sudo mv vcluster /usr/local/bin
- Create a virtual cluster:
vcluster create my-vcluster
- Connect to the virtual cluster:
vcluster connect my-vcluster
- Use kubectl to interact with the virtual cluster:
kubectl get pods
- Delete the virtual cluster when done:
vcluster delete my-vcluster
For more advanced usage and configuration options, refer to the official vcluster documentation.
Competitor Comparisons
Home for Cluster API, a subproject of sig-cluster-lifecycle
Pros of Cluster API
- Provides a standardized, declarative approach to cluster lifecycle management across various infrastructure providers
- Offers more extensive support for managing multiple clusters and complex multi-cloud deployments
- Has strong community support and is part of the official Kubernetes SIG ecosystem
Cons of Cluster API
- Requires more setup and configuration compared to vcluster's lightweight approach
- Has a steeper learning curve, especially for users new to Kubernetes cluster management
- May be overkill for simple use cases or local development environments
Code Comparison
vcluster:
apiVersion: v1
kind: ConfigMap
metadata:
name: my-vcluster-config
data:
config.yaml: |
sync:
nodes: false
Cluster API:
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: my-cluster
spec:
clusterNetwork:
pods:
cidrBlocks: ["192.168.0.0/16"]
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSCluster
name: my-cluster
The code examples demonstrate the difference in complexity and scope between the two projects. vcluster focuses on lightweight virtual clusters within an existing Kubernetes cluster, while Cluster API provides a more comprehensive approach to managing full-fledged Kubernetes clusters across various infrastructure providers.
Little helper to run CNCF's k3s in Docker
Pros of k3d
- Lightweight and fast setup for local Kubernetes development
- Supports multi-node clusters, simulating more realistic environments
- Integrates well with existing Docker infrastructure
Cons of k3d
- Limited to local development environments
- Doesn't provide true multi-tenancy or isolation between virtual clusters
- May not fully replicate production Kubernetes environments
Code Comparison
k3d:
k3d cluster create mycluster --servers 3 --agents 2
vcluster:
vcluster create mycluster
kubectl config use-context vcluster_mycluster
Key Differences
- k3d creates lightweight Kubernetes clusters using k3s in Docker containers, ideal for local development and testing.
- vcluster creates virtual clusters within existing Kubernetes clusters, providing better isolation and multi-tenancy.
- k3d is more focused on local development, while vcluster can be used in both local and production environments.
- vcluster offers better resource isolation and namespace separation between virtual clusters.
- k3d provides an easier setup for multi-node clusters, which can be beneficial for testing distributed applications.
Both tools serve different purposes and can be complementary in a Kubernetes development workflow, with k3d excelling in local development and vcluster offering more flexibility for multi-tenancy and production-like environments.
Run Kubernetes locally
Pros of Minikube
- Simulates a full Kubernetes cluster on a local machine, providing a more realistic environment
- Supports multiple hypervisors and operating systems, offering greater flexibility
- Includes built-in addons for enhanced functionality and easier management
Cons of Minikube
- Requires more system resources due to running a full VM
- Setup and teardown can be slower compared to lightweight alternatives
- Limited scalability for testing larger deployments or multi-node scenarios
Code Comparison
Minikube:
minikube start
kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10
kubectl expose deployment hello-minikube --type=NodePort --port=8080
minikube service hello-minikube
vcluster:
vcluster create my-vcluster
vcluster connect my-vcluster
kubectl create deployment hello-vcluster --image=k8s.gcr.io/echoserver:1.10
kubectl expose deployment hello-vcluster --type=ClusterIP --port=8080
kubectl port-forward service/hello-vcluster 8080:8080
The code comparison shows that both tools allow for creating and interacting with Kubernetes deployments, but vcluster operates within an existing cluster, while Minikube creates a standalone environment. Minikube's approach is more straightforward for local development, while vcluster offers more flexibility for working within existing infrastructure.
Lightweight Kubernetes
Pros of k3s
- Lightweight and resource-efficient, suitable for edge computing and IoT devices
- Includes built-in storage and load balancing solutions
- Easier to set up and manage for standalone Kubernetes clusters
Cons of k3s
- Limited flexibility in terms of customization compared to vcluster
- May not be ideal for creating multiple virtual clusters within a single host cluster
- Less suitable for development and testing environments that require isolation
Code Comparison
k3s installation:
curl -sfL https://get.k3s.io | sh -
vcluster creation:
vcluster create my-vcluster
Key Differences
- k3s is a lightweight Kubernetes distribution, while vcluster creates virtual clusters within existing Kubernetes clusters
- k3s is better suited for production environments, especially in resource-constrained scenarios
- vcluster excels in creating isolated development and testing environments within a shared cluster
Use Cases
- k3s: Edge computing, IoT devices, small-scale production deployments
- vcluster: Development environments, testing, multi-tenancy scenarios in shared clusters
Community and Ecosystem
- k3s has a larger community and ecosystem, being part of the CNCF landscape
- vcluster is gaining traction for its unique approach to virtual Kubernetes clusters
Kubernetes IN Docker - local clusters for testing Kubernetes
Pros of kind
- Creates a full, isolated Kubernetes cluster, ideal for testing and development
- Supports multi-node clusters, allowing for more realistic testing scenarios
- Integrates well with CI/CD pipelines and automated testing frameworks
Cons of kind
- Requires more system resources due to running full nodes
- Slower to start up compared to lightweight alternatives
- Limited flexibility in customizing the underlying host environment
Code Comparison
kind:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
vcluster:
apiVersion: v1
kind: ConfigMap
metadata:
name: my-vcluster-config
data:
values.yaml: |
sync:
nodes:
enabled: true
Key Differences
vcluster creates virtual clusters within an existing Kubernetes cluster, while kind creates standalone clusters using Docker containers. vcluster is more lightweight and faster to start, but kind provides a more isolated and complete cluster environment. vcluster allows for easier integration with existing cluster resources, while kind offers better isolation for testing purposes.
Both tools serve different use cases and can be valuable in different scenarios. kind is better suited for full cluster testing and CI/CD pipelines, while vcluster excels in multi-tenancy scenarios and rapid development environments within existing clusters.
Backup and migrate Kubernetes applications and their persistent volumes
Pros of Velero
- Broader scope: Velero provides comprehensive backup and restore capabilities for entire Kubernetes clusters, including persistent volumes.
- Mature project: Velero has been around longer and has a larger community, potentially offering more stability and support.
- Cloud provider integration: Velero supports various cloud providers for storing backups, offering flexibility in deployment scenarios.
Cons of Velero
- Resource intensive: Velero can be more resource-heavy compared to vcluster, especially when backing up large clusters.
- Complexity: Setting up and configuring Velero can be more complex, particularly for smaller deployments or simpler use cases.
- Limited multi-tenancy: While Velero can back up namespaces, it doesn't provide the same level of multi-tenancy isolation as vcluster.
Code Comparison
Velero backup creation:
apiVersion: velero.io/v1
kind: Backup
metadata:
name: example-backup
spec:
includedNamespaces:
- default
vcluster creation:
apiVersion: cluster.loft.sh/v1
kind: VirtualCluster
metadata:
name: example-vcluster
spec:
isolation:
enabled: true
Both projects serve different primary purposes, with Velero focusing on backup and disaster recovery, while vcluster emphasizes creating isolated virtual clusters within a host cluster. The code examples reflect these different focuses, with Velero defining a backup operation and vcluster specifying a virtual cluster configuration.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Website ⢠Quickstart ⢠Documentation ⢠Blog ⢠Twitter ⢠Slack
Virtual clusters are fully functional Kubernetes clusters nested inside a physical host cluster providing better isolation and flexibility to support multi-tenancy. Multiple teams can operate independently within the same physical infrastructure while minimizing conflicts, maximizing autonomy, and reducing costs.
Virtual clusters run inside host cluster namespaces but function as separate Kubernetes clusters, with their own API server, control plane, syncer, and set of resources. While virtual clusters share the physical resources of the host cluster (such as CPU, memory, and storage), they manage their resources independently, allowing for efficient utilization and scaling.
Virtual clusters interact with the host cluster for resource scheduling and networking but maintain a level of abstraction to ensure operations within a virtual cluster don't directly affect the host cluster's global state.
âï¸ Do you like vcluster? Support the project with a star âï¸
Benefits
Virtual clusters provide immense benefits for large-scale Kubernetes deployments and multi-tenancy.
Robust security and isolation
- Granular Permissions: vCluster users operate with minimized permissions in the host cluster, significantly reducing the risk of privileged access misuse. Within their vCluster, users have admin-level control, enabling them to manage CRDs, RBAC, and other security policies independently.
- Isolated Control Plane: Each vCluster comes with its own dedicated API server and control plane, creating a strong isolation boundary.
- Customizable Security Policies: Tenants can implement additional vCluster-specific governance, including OPA policies, network policies, resource quotas, limit ranges, and admission control, in addition to the existing policies and security measures in the underlying physical host cluster.
- Enhanced Data Protection: With options for separate backing stores, including embedded SQLite, etcd, or external databases, virtual clusters allow for isolated data management, reducing the risk of data leakage between tenants.
Access for tenants
- Full Admin Access per Tenant: Tenants can freely deploy CRDs, create namespaces, taint, and label nodes, and manage cluster-scoped resources typically restricted in standard Kubernetes namespaces.
- Isolated yet Integrated Networking: While ensuring automatic isolation (for example, pods in different virtual clusters cannot communicate by default), vCluster allows for configurable network policies and service sharing, supporting both separation and sharing as needed.
- Node Management: Assign static nodes to specific virtual clusters or share node pools among multiple virtual clusters, providing flexibility in resource allocation.
Cost-effectiveness and reduced overhead
- Lightweight Infrastructure: Virtual clusters are significantly more lightweight than physical clusters, able to spin up in seconds, which contrasts sharply with the lengthy provisioning times often seen in environments like EKS (~45 minutes).
- Resource Efficiency: By sharing the underlying host cluster's resources, virtual clusters minimize the need for additional physical infrastructure, reducing costs and environmental impact.
- Simplified Management: The vCluster control plane, running inside a single pod, along with optional integrated CoreDNS, minimizes the operational overhead, making virtual clusters especially suitable for large-scale deployments and multi-tenancy scenarios.
Enhanced flexibility and compatibility
- Diverse Kubernetes Environments: vCluster supports different Kubernetes versions and distributions (including K8s, K3s, and K0s), allowing version skews. This makes it possible to tailor each virtual cluster to specific requirements without impacting others.
- Adaptable Backing Stores: Choose from a range of data stores, from lightweight (SQLite) to enterprise-grade options (embedded etcd, external data stores like Global RDS), catering to various scalability and durability needs.
- Runs Anywhere: Virtual clusters can run on EKS, GKE, AKS, OpenShift, RKE, K3s, cloud, edge, and on-prem. As long as it's a K8s cluster, you can run a virtual cluster on top of it.
Improved scalability
- Reduced API Server Load: Virtual clusters, each with their own dedicated API server, significantly reduce the operational load on the host cluster's Kubernetes API server by isolating and handling requests internally.
- Conflict-Free CRD Management: Independent management of CRDs within each virtual cluster eliminates the potential for CRD conflicts and version discrepancies, ensuring smoother operations and easier scaling as the user base expands.
Common use cases
Pre-production
- Empower developers with self-service Kubernetes: Simplify Kubernetes access for developers through self-service virtual clusters, reducing human error and enhancing developer autonomy without compromising security and compliance requirements.
- Accelerate CI/CD with ephemeral Kubernetes clusters: Instantly create clean, new virtual Kubernetes clusters for each pull request, enabling fast, isolated testing and PR previews without wait times and the struggles of a shared test environment.
Production
- Elevate your ISV offering with a dedicated cluster per customer: Host each customer in a virtual cluster with strict tenant isolation and seamless scalability, while consolidating essential tools into a unified platform stack serving multiple tenants.
- Build a managed Kubernetes service with best-in-class COGS and high margins: Enable direct customer access to dedicated virtual Kubernetes clusters, streamlining node and resource allocation for industry-leading efficiency and unparalleled scalability.
Quick Start
Refer to our quick start guide to deploy your first vCluster!
Contributing
Thank you for your interest in contributing! Please refer to CONTRIBUTING.md for guidance.
License
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Top Related Projects
Home for Cluster API, a subproject of sig-cluster-lifecycle
Little helper to run CNCF's k3s in Docker
Run Kubernetes locally
Lightweight Kubernetes
Kubernetes IN Docker - local clusters for testing Kubernetes
Backup and migrate Kubernetes applications and their persistent volumes
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot