AWS EBS as persistent storage for Openshift

Openshift supports different persistent storage backends. Ceph, gluster, Amazon EBS, just to name some of them. In using ceph as persistent storage for openshift pods blog post I described how to hook openshift installation to use CEPH cluster as persistent storage for openshift pods. Here will be described process how configure Openshift to use EBS AWS based storage as persistent volumes for openshift pods

It is assumed here that as starting point is working Openshift environment. In order to use Amazon EBS storage with openshift, it is necessary to do small changes in openshift configuration

  • Openshift master and all Openshift nodes need to be configured to support AWS EBS. Openshift documentation gives nice overview of necessary changes
  • After configuring openshift master/nodes to be able to reach / allocate EBS, next steps are straightforward.
    Examples of PersistentVolume , PersistentVolumeClaim, and Pod json file are showed below

  • PersistenVolume
  • PersistentVolumeClaim
  • pod file
  • In above .json files is necessary to adapt them with proper pv name, size, pvc name, size, pod name and etc to correspond environment where used. After adapting them, create PersistentVolume and PersistentVolumeClaim

    # oc create -f pv.json 
    # oc create -f pvc.json 
     # oc get pv
    NAME       LABELS    CAPACITY     ACCESSMODES     STATUS    CLAIM               REASON      AGE
    pvolume1               1          RWO             Bound     default/pvclaim1                3h
    # oc get pvc 
    pvclaim1              Bound     pvolume1    1             RWO             3h

    Now we can create pod which will use above created PersistentVolume and PersistentVolumeClaim

    # oc create -f pod.json
    # oc get pods | grep pod1
    pod1                      1/1       Running   0          30s

    Now, on node where pod runs, we will see

    # fdisk -l | grep xvd
    WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.
    Disk /dev/xvda: 10.7 GB, 10737418240 bytes, 20971520 sectors
    Disk /dev/xvdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
    /dev/xvdb1            2048    20971519    10484736   8e  Linux LVM
    Disk /dev/xvdf: 1073 MB, 1073741824 bytes, 2097152 sectors ---- this is EBS device used inside pod 

    /dev/xvdf is device mapped to node, I know this as it is only 1GB device on my system. Further, device is visible in mount output

    # mount | grep xvdf
    /dev/xvdf on /var/lib/origin/openshift.local.volumes/plugins/ type ext4 (rw,relatime,seclabel,data=ordered)
    /dev/xvdf on /var/lib/origin/openshift.local.volumes/pods/3a467bfd-eeb7-11e5-9e13-02982d3b6367/volumes/ type ext4 (rw,relatime,seclabel,data=ordered)

    it is mounted at mount point under /var/lib/origin/openshift.local.volumes …

    If we take closer look, we will see there, EBS volume id ( vol-3f7210c ), then plugin used ( aws-ebs ) – this can be useful if it is necessary to do sorting / searching devices allocated on same Openshift node and commit from two different storage backennds ( eg, EBS and CEPH )
    In pod we see

     # oc exec  pod1 -- mount  | grep xvdf
    /dev/xvdf on /mnt/persistentvolume type ext4 (rw,relatime,seclabel,data=ordered)

    For every pod using EBS as persistant storage pair of lines similar to ones above will be visible on Openshift node


    #amazon-ebs, #linux, #openshift, #persistantvolume, #persistantvolumeclaim, #pod, #storage