Kubestr로 Storage 구성 및 성능 평가

Kubestr

https://kubestr.io/

Kubernetes에서는 다양한 Storage 를 연동할 수 있도록 CSI (Container Storage Interface) 가 존재하고, CSI Driver를 클러스터에 설치하여 다양한 벤더의 Stroage 를 연동하여 사용할 수 있다.

 

Kubestr은 kubernetes 스토리지 옵션을 검색, 검증 및 평가하기 위한 도구 모음이다. 

 

주요기능

• 클러스터에 있는 다양한 스토리지 옵션을 식별
• 스토리지 옵션이 올바르게 구성되었는지 확인
• FIO와 같은 일반적인 벤치마킹 도구를 사용하여 스토리지를 평가
• graphical filesystem browser에서 PersistentVolumeClaim의 내용을 확인

 

설치

 

# wget https://github.com/kastenhq/kubestr/releases/download/v0.4.31/kubestr_0.4.31_Linux_amd64.tar.gz

# tar xvfz kubestr_0.4.31_Linux_amd64.tar.gz
# chmod +x kubestr
# mv kubestr /usr/local/bin/.

 

실행을 해보면 아래와 같다.

# kubestr

**************************************
  _  ___   _ ___ ___ ___ _____ ___
  | |/ / | | | _ ) __/ __|_   _| _ \
  | ' <| |_| | _ \ _|\__ \ | | |   /
  |_|\_\\___/|___/___|___/ |_| |_|_\

Explore your Kubernetes storage options
**************************************
Kubernetes Version Check:
  Valid kubernetes version (v1.21.7)  -  OK

RBAC Check:
  Kubernetes RBAC is enabled  -  OK

Aggregated Layer Check:
  The Kubernetes Aggregated Layer is enabled  -  OK

Available Storage Provisioners:

  nfs-provisioner:
    Unknown driver type.

    Storage Classes:
      * my-storage

    To perform a FIO test, run-
      ./kubestr fio -s <storage class>
      
  openebs.io/local:
    Unknown driver type.

    Storage Classes:
      * openebs-device
      * openebs-hostpath

    To perform a FIO test, run-
      ./kubestr fio -s <storage class>

* my-stroage는 현재 nfs provisioner csi driver이다.

 

한번 성능을 측정해 보자.

# kubestr fio -s my-storage
Elapsed time- 3m15.162333148s
FIO test results:

FIO version - fio-3.20
Global options - ioengine=libaio verify=0 direct=1 gtod_reduce=1
WW
JobName: read_iops
  blocksize=4K filesize=2G iodepth=64 rw=randread
read:
  IOPS=120.791351 BW(KiB/s)=498
  iops: min=14 max=204 avg=126.774193
  bw(KiB/s): min=56 max=816 avg=507.225800
WW
JobName: write_iops
  blocksize=4K filesize=2G iodepth=64 rw=randwrite
write:
  IOPS=202.905685 BW(KiB/s)=827
  iops: min=20 max=404 avg=205.937500
  bw(KiB/s): min=80 max=1616 avg=823.875000
WW
JobName: read_bw
  blocksize=128K filesize=2G iodepth=64 rw=randread
read:
  IOPS=199.081451 BW(KiB/s)=25976
  iops: min=34 max=394 avg=203.125000
  bw(KiB/s): min=4352 max=50432 avg=26004.562500
WW
JobName: write_bw
  blocksize=128k filesize=2G iodepth=64 rw=randwrite
write:
  IOPS=96.393646 BW(KiB/s)=12831
  iops: min=6 max=202 avg=97.937500
  bw(KiB/s): min=768 max=25856 avg=12541.218750
WW
Disk stats (read/write):
  -  OK

Disk stats (read/write):
  -  OK

 

이번에는 OpenEBS 를 테스트해 보자.

# kubestr fio -s openebs-hostpath

Elapsed time- 39.95225191s
FIO test results:

FIO version - fio-3.20
Global options - ioengine=libaio verify=0 direct=1 gtod_reduce=1
WW
JobName: read_iops
  blocksize=4K filesize=2G iodepth=64 rw=randread
read:
  IOPS=1452.786255 BW(KiB/s)=5827
  iops: min=768 max=2048 avg=1454.266724
  bw(KiB/s): min=3072 max=8192 avg=5817.100098
WW
JobName: write_iops
  blocksize=4K filesize=2G iodepth=64 rw=randwrite
write:
  IOPS=287.459167 BW(KiB/s)=1166
  iops: min=122 max=640 avg=292.799988
  bw(KiB/s): min=488 max=2560 avg=1171.333374
WW
JobName: read_bw
  blocksize=128K filesize=2G iodepth=64 rw=randread
read:
  IOPS=1134.326416 BW(KiB/s)=145730
  iops: min=622 max=1450 avg=1136.333374
  bw(KiB/s): min=79616 max=185600 avg=145457.296875
WW
JobName: write_bw
  blocksize=128k filesize=2G iodepth=64 rw=randwrite
write:
  IOPS=290.822327 BW(KiB/s)=37752
  iops: min=138 max=660 avg=294.133331
  bw(KiB/s): min=17664 max=84480 avg=37654.566406
WW
Disk stats (read/write):
  dm-0: ios=44395/9496 merge=0/0 ticks=447780/2149566 in_queue=2661641, util=100.000000%
  sda: ios=44394/9496 merge=1/0 ticks=414274/2147993 in_queue=2562419, util=99.448715%
  -  OK