Skip to content

camper42/juicefs-csi-driver

 
 

Repository files navigation

JuiceFS CSI Driver

Build Status

The JuiceFS Container Storage Interface (CSI) Driver implements the CSI specification for container orchestrators to manage the lifecycle of JuiceFS filesystems.

Installation

Deploy the driver:

kubectl apply -f https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/deploy/k8s.yaml

Additional steps could be required on some provider, e.g. Aliyun Container Service Kubernetes. See Troubleshooting#AttachVolume.Attach failed for details.

Upgrade

We have two components to upgrade:

  • CSI Driver
  • JuiceFS client in CSI Driver

Upgrade CSI Driver

  1. Stop all pods using this driver.
  2. Upgrade driver:
    • If you're using latest tag, simple run kubectl rollout restart -f k8s.yaml and make sure juicefs-csi-controller and juicefs-csi-node pods are restarted.
    • If you have pinned to a specific version, modify your k8s.yaml to a newer version, then run kubectl apply -f k8s.yaml.

Visit here for more versions.

Upgrade JuiceFS client

Refer to the notes in Examples section below.

Examples

Before the example, you need to:

  • Get yourself familiar with how to setup Kubernetes and how to create JuiceFS filesystem.
  • When creating JuiceFS filesystem, make sure it is accessible from Kuberenetes cluster. It is recommended to create the filesystem inside the same region as Kubernetes cluster.
  • Install JuiceFS CSI driver following the Installation steps.

Example links

Notes:

  • Since JuiceFS is an elastic filesystem it doesn't really enforce any filesystem capacity. The actual storage capacity value in persistence volume and persistence volume claim is not used when creating the filesystem. However, since the storage capacity is a required field by Kubernetes, you must specify the value and you can use any valid value e.g. 10Pi for the capacity.
  • Automatically upgrade of JuiceFS Client in JuiceFS CSI Driver image is now supported, and can be disabled. JuiceFS Client will upgrade automatically everytime before mounting a PersistentVolume if enabled. We also offer different ways to control this function:
    • If you are using JuiceFS CSI Driver image with latest tag, then auto-upgrade is enabled by default. JuiceFS Client will try to upgrade itself everytime before mounting, a very short check time will be taken if no upgrade.

    • If you have pinned JuiceFS CSI Driver to a specific version, then auto-upgrade is disabled by default. You can still enable this through environment variables:

      • JFS_AUTO_UPGRADE: enable auto-upgrade if set to none empty value, otherwise disable
      • JFS_AUTO_UPGRADE_TIMEOUT: the timeout for auto-upgrade in seconds(default 10)

      configure the above environment variables when deploy JuiceFS CSI Driver. Notice JuiceFS CSI Driver need to be re-deployed to ensure the environment variables are applied to controller and node driver.

CSI Specification Compatibility

JuiceFS CSI Driver \ CSI Version v0.3 v1.0
master branch (csi-v1) no yes
csi-v0 branch yes no

Interfaces

Currently only static provisioning is supported. This means an JuiceFS filesystem needs to be created manually on juicefs web console first. After that it can be mounted inside a container as a volume using the driver.

The following CSI interfaces are implemented:

  • Node Service: NodePublishVolume, NodeUnpublishVolume, NodeGetCapabilities, NodeGetInfo, NodeGetId
  • Identity Service: GetPluginInfo, GetPluginCapabilities, Probe

JuiceFS CSI Driver on Kubernetes

The following sections are Kubernetes specific. If you are a Kubernetes user, use this for driver features, installation steps and examples.

Kubernetes Version Compatibility Matrix

JuiceFS CSI Driver \ Kubernetes Version v1.11 v1.12 v1.13 v1.14 v1.15 v1.16
master branch (csi-v1) no no yes yes yes yes
csi-v0 branch yes yes yes yes

Container Images

JuiceFS CSI Driver Version Image
master branch juicedata/juicefs-csi-driver:latest
csi-v1 branch juicedata/juicefs-csi-driver:csi-v1
csi-v0 branch juicedata/juicefs-csi-driver:csi-v0

Features

  • Static provisioning - JuiceFS filesystem needs to be created manually first, then it could be mounted inside container as a persistent volume (PV) using the driver.
  • Mount options - CSI volume attributes can be specified in the persistence volume (PV) to define how the volume should be mounted.
  • Read write many - Support ReadWriteMany access mode
  • Sub path - provision persisten volume with subpath in JuiceFS filesystem
  • Dynamic provisioning - allows storage volumes to be created on-demand
Feature \ JuiceFS CSI Driver master (csi-v1) csi-v0
static provisioning yes yes
mount options yes yes
read write many yes yes
sub path yes yes
dynamic provisioning yes no

Validation

JuiceFS CSI driver has been validated in the following Kubernetes version

Kubernetes \ JuiceFS CSI Driver master (csi-v1) csi-v0
v1.11.9 / kops 1.11.1 yes
v1.12.6-eks-d69f1b / AWS EKS yes
v1.12.6-aliyun.1 / Aliyun CS K8s yes
v1.13.0 / minikube 1.4.0 yes
v1.13.5 / kops 1.13.0-alpha.1 yes
v1.14.1 / kops (git-39884d0b5) yes yes
v1.14.8 / minikube 1.4.0 yes
v1.15.0 / minikube 1.4.0 yes
v1.16.0 / minikube 1.4.0 yes
v1.17.2 / minikube 1.7.2 yes

Manual configuration is required for Aliyun Container Service Kubernetes. See Troubleshooting#AttachVolume.Attach failed for details.

Known issues

JuiceFS CSI volumes can NOT reconcile #14

When juicefs-csi-node is killed, existing JuiceFS volume will become inaccessible. It will not recover automatically even after juicefs-csi-node reconcile.

Delete the pods mounting JuiceFS volume and recreate them to recover.

Develop

See DEVELOP document.

License

This library is licensed under the Apache 2.0 License.

Packages

No packages published

Languages

  • Go 79.1%
  • Shell 9.9%
  • Makefile 6.8%
  • Dockerfile 4.2%