Search code examples
rancherrancher-rkelonghorn

Longhorn using more than 50% of storage as reserved space


I'm running Longhorn v1.2.3 on RKE1 cluster (provisioned by rancher), this cluster has 5 nodes with dedicated 20GiB disks mounted on /var/lib/longhorn, with ext4 filesystem and 0% reserved blocks for root user/group.

In the dashboard, i see the following stats:

Type Size
Schedulable 33.5 Gi
Reserved 58.1 Gi
Used 6.18 Gi
Disabled 0 Bi
Total 97.8 Gi

I changed Storage Minimal Available Percentage in settings to 5 (from 25 as i recall), but that haven't changed anything. When i open "nodes" tab, i see the following in "Size" tab:

7.86 Gi
+11.7 Gi Reserved

exact size varies for different nodes, but it's around 8Gi.

These dedicated disks were added after provisioning longhorn in cluster, and system disks are 40 GiB in size, so possibly the reason for this overuse is because reserved size was calculated at the time, when longhorn was deployed alongside operating system, and haven't adjusted when i mounted this folder to new disk.

Why do i have more than half of my space "reserved"? What can i do to get more usable space from longhorn? Thanks!


Solution

  • After digging deeper and finding that it was one day possible to adjust these values from UI (i wasn't able to find it), i've searched for longhorn CRDs, and came across nodes.longhorn.io. And inside definition i've found exactly what i searched for:

    spec:
      allowScheduling: true
      disks:
        default-disk-fd0000000000:
          allowScheduling: true
          evictionRequested: false
          path: /var/lib/longhorn/
          storageReserved: 536870912
          tags: null
    

    Here i changed storageReserved to 536870912 (512 MiB) on all nodes, just in case, and longhorn applied this change immediately. This is ok in my case, because those disk are dedicated, and, per docs

    Reserved: The space reserved for other applications and system.

    Now i have my storage back, hope it helps

    Edit: I've found a reason why i wasn't able to find a GUI setting - simply because, due to sidebar in my browser, it was hidden by not-that-obvious horizontal scroll bar.