Linux @lemmy.ml unhinge @programming.dev 7 mo. ago

Right way to check free/used storage of btrfs filesystem [SOLVED]

While checking for used and free space in a btrfs subvolume, I'm not getting a consistent value. It's confusing and doesn't help.

What is the correct way to find used/free space?
Why are these values inconsistent (except normal du)?

According to btrfs fi usage /home, 83.21 GiB is used.

Overall:
    Device size:		 149.98GiB
    Device allocated:		 100.07GiB
    Device unallocated:		  49.91GiB
    Device missing:		     0.00B
    Device slack:		     0.00B
    Used:			  83.21GiB
    Free (estimated):		  63.06GiB	(min: 38.10GiB)
    Free (statfs, df):		  63.06GiB
    Data ratio:			      1.00
    Metadata ratio:		      2.00
...

As per btrfs fi df /home, used space is 82.86 GiB, not 83.21 GiB.

Data, single: total=96.01GiB, used=82.86GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=2.00GiB, used=178.61MiB
GlobalReserve, single: total=99.50MiB, used=0.00B

As per btrfs fi du -s /home , used space is 63.11 GiB.

     Total   Exclusive  Set shared  Filename
  63.11GiB    13.64GiB    49.01GiB  /home

While according to du -hs /home, 64GiB is used.

Also, maximum space used should be close to 72 GiB as per btrfs fi du -s / and 73 GiB as per du -hs /, if btrfs fi usage includes all subvolumes . '/home' and '/' are on separate subvolumes.

11 comments

As per btrfs fi df /home, used space is 82.86 GiB, not 83.21 GiB.

That's just used data. The global used metric likely incorporates metadata etc. too. System aswell as the GlobalReserve are probably accounted as fully used as they're, well, reserved.

As per btrfs fi du -s /home , used space is 63.11 GiB.

Total Exclusive Set shared Filename 63.11GiB 13.64GiB 49.01GiB /home

While according to du -hs /home, 64GiB is used.

Likely compression or inline extents. btrfs only reports apparent size to du and friends unfortunately.

Also, maximum space used should be close to 72 GiB as per btrfs fi du -s / and 73 GiB as per du -hs /, if btrfs fi usage includes all subvolumes . '/home' and '/' are on separate subvolumes.

Your home has a lot of shared extents which indicates to me that you have at least one snapshot of it.

You also wrote 13.6GiB of new data to your home since the snapshot. Assuming a similar amount of data was deleted/overwritten since, that would add up to 76GiB. If there's perhaps one or two more snapshots, that would explain the rest.

Snapshots are "free" only so long as you don't write or delete any data in the origin.
- Thanks that helped. I have one snapshot of home. ~~Size of diff between btrfs subvolume and~~ Additional space used by snapshot is 11GiB (probably) and btrfs fi du -s / is 72GiB, making 83GiB (closer to btrfs fi df /).
  
  Size of diff between btrfs subvolume and snapshot is 11GiB
  
  WDYM by "diff"?
  
  Also forgot to mention but if you want to know what's taking how much space on your btrfs, try btdu. It uses a sampling-based approach and will therefore never be 100% accurate but it should be quite accurate enough after a little bit.
I'm not an expert on btrfs, but I assume the inconsistencies come from deduplication, metadata, and maybe compression. I think some of them just count raw block storage, and some include the cost of metadata.

Traditional du assumes that each file takes up it's full space on disk which isn't always the case on btrfs. When using btrfs backed oci images, storage can easily appear multiple times higher.

I use btrfs filesystem usage /. I'm not sure that it is the "correct" way, but it works fairly well.
- Between btrfs filesystem usage / and btrfs filesystem du -s / there's nearly 11GiB difference for used space. I have checked btrfs du -hs <path/to/subvolume> for all subvolume in the filesystem, and total seems to be 72 GiB, hence the confusion. Still I don't know if I'm using the tools properly or something else is at fault here.
  
  To correct myself, 11GiB is additional space used by snapshot probably ~~used space difference between btrfs fi usage and btrfs fi du -s / is because of diff between snapshot and parent volume (didn't consider that while adding all used GiB of subvolumes)~~. So btrfs filesystem usage works well to check used/free space.
  
  edit: fix incorrect args; additional space is not diff
It'd be really nice if df -h actually worked for btrfs...

Also, the discrepancies can be way worse than that.

Which is not to say don't use it, it's a way superior filesystem, just needs some polishing with these things. Also, remember to occasionally sudo btrfs balance, or better, set up a systemd to do so...
I needed to find large directories on disk the other week, and found the tool btdu to be quite useful.

https://github.com/CyberShadow/btdu
Masbe try compsize:
https://github.com/kilobyte/compsize

You've viewed 11 comments.