[btrfs] Tips and Tricks

[anon23612428]

BTRFS - Tips and tricks

I thought that Btrfs doesn't get all too much love here, so I decided to show you a few things you can do with btrfs.
I do not go into technical details all too much, as this is mainly meant for curious newcomers to Btrfs.

Please also note that most of the commands below need sudo, I haven't written it down explicitly.

Do not hesitate to correct mistakes or make suggestions for additional tips and tricks.

Further information can be found in the usual places, i.e. Wikipedia, Arch Wiki, official Btrfs Wiki, and of course the manpages.

mkfs.btrfs - filesystem creation

Generally you do not need any additional options to pass to mkfs.btrfs, as the defaults are fine.

However it's recommended to pass a label regardless of the filesystem; just use the -L name-of-label parameter.

By default, the "single" profile is used for both data and metadata, except on rotational drives where metadata is duplicated (profile name "dup") for performance reasons (less head seeks).

Also note that btrfs supports partitionless drives, i.e. instead of formatting a partition, you can format the entire block device.

This is NOT recommended for root due to possible problems with GRUB, but for other purposes it is actually helpful, because you don't have to mess with partitions anymore, thus eliminating a layer of complexity.

You can still create "subdivisions" with subvolumes.

Btrfs has its own RAID support.
If you have two empty drives you can very easily create a RAID1 with
mkfs.btrfs -d raid1 -m raid1 /dev/sda /dev/sdb
For partitions, just replace /dev/sda with /dev/sdaX and /dev/sdb with /dev/sdbX.

Mount options

There are a few interesting btrfs specific mount options:

• compress=lzo|zlib|zstd: this enables compression for newly created files. Three algorithms are available: zlib, lzo and zstd. Zstd needs at least kernel 4.14 and currently doesn't work on root because Manjaro's GRUB lacks support.

• autodefrag: as the names implies, this enables automatic defragmentation. Not needed for SSDs.

• ssd and nossd: btrfs autodetects SSDs based on the sysfs "rotational" entry. You can however still explicitly enable or disable SSD optimisation with these options.

• subvol and subvolid: enables mounting of a specific subvolume, either by passing the path or the subvolume ID.

There are more mount options but those are out of scope and generally not needed.

btrfs filesystem usage

You certainly are used to using the df command to show free space of a partition with conventional filesystems.

However df is not very accurate for btrfs partitions.

Instead, it's highly recommended to use btrfs filesystem usage /path instead, which gives detailed info about filesystem usage.

Expand space

Imagine you have your /home on btrfs and you start running out of space.

You decide to get a new drive to have more free space available.

With btrfs, it's very easy to add that new drive to your /home.

1. Create an unformatted partition on the new drive, let's assume it's /dev/sdb1.

2. Run the following command:
   btrfs device add /dev/sdb1 /home

3. Now just issue btrfs filesystem usage /home and verify that free and total space have increased accordingly.

4. You can now just continue using /home as always, but since you have two drives now, you can add a little redundancy. You can use RAID1 for metadata only with the following command:
   btrfs balance start -mconvert=raid1 /home

This will mirror the metadata on each drive, so if metadata on one drive gets corrupted, btrfs will automatically use metadata from the good drive. You will lose a little space, but not much.

Reflink copying

Due to the copy-on-write nature of btrfs, you can easily create copies of existing files which initially do not occupy additional free space (until modified).

This also means that the copy is created near instantly.

This can be done by using the --reflink parameter with cp, i.e. cp --reflink source destination.

Disabling COW (copy-on-write) for files and directories

In some cases it is useful to disable COW, for example for partitions that hold VirtualBox VDIs.

This can be achieved with chattr +C name-of-file (or name-of-directory).

Note that it will only be disabled for newly created files, not for existing files!

If set on an (empty) directory, it will apply to all newly created files within that directory.

Subvolumes and Snapshots

Let's say that you mounted your newly created btrfs partition /dev/sdb1 to /mnt.

You can now create a subvolume with btrfs subvolume create /mnt/name-of-subvolume, and list all the subvolumes of that path with btrfs subvolume list -ap /mnt, and the output could be as follows

ID 260 gen 13 parent 5 top level 5 path name-of-subvolume

Note the "parent" ID number: the root parent ID is always 5, this is the default (root) subvolume for every btrfs partition that is active by default.

Let's create a few files in the first subvolume:

cd /mnt/name-of-subvolume && for i in {1..9}; do touch file_$i; done

And now, let's create a snapshot of that subvolume with btrfs subvolume snapshot /mnt/name-of-subvolume /mnt/snapshot1. (Pass the -r option to create a read-only snapshot.)

Now delete one of the files we just created with rm file_4.

Oh noz! That was an important file!

Don't despair, you have a snapshot and you can recover file_4. Just go to /mnt/snapshot1, the location where the snapshot was created, and you'll see that file_4 is there. You can just copy it back.

Another possibility is to mount the snapshot directly.

For that, you need the ID of the snapshot which you can get with the list command as mentioned above.

Then just mount it by passing the subvolid option with

mount /dev/sdb1 /mnt2 -o subvolid=262

The snapshot is now available under /mnt2.

Note that only subvolumes can be snapshot.

No fsck??

Btrfs currently has no fully working fsck, but due to the nature of btrfs, it isn't as strictly needed as with conventional filesystems.
The name of the btrfs fsck utility is btrfs check -p /dev/sdb1.
While it can detect problems, it cannot (yet) repair them. That's why by default it is run in read-only mode, no data being modified.
You can force enable the repair mode with --repair, but it is highly dangerous! Do not use!

Btrfs uses checksums for both data and metadata.
In order to verify the checksums, the command btrfs scrub is used (see below).

Balance and Scrub

See here: [btrfs] Balance & Scrub

Reliability

Difficult subject.

I've been using btrfs since its early days, when it was still considered "beta", and in all those years, I haven't had a single data loss despite several power outages and hard freezes.
However, other people haven't had as much luck.

Btrfs is still under heavy development and while the on-disk format has been finalised since years, many features like transparent encryption and in-band deduplication are not implemented yet. RAID5/6 are implemented but experimental and should not be used.

There are two recovery utilities provided by btrfs-utils: btrfs rescue and btrfs restore.
The first one can be used to recover superblocks and the chunk tree, the second one to recover lost files.

Send and Receive

btrfs is able to create a binary diff between two snapshots with btrfs send, and which can be replayed (or "patched") with btrfs receive.
As I haven't used these features yet, I cannot say much about it, but now you know that the feature exists. Edit (by @eugen-b): Just have a look at the post about send|receive by @TomZ below.

Debug information

btrfs also provides a utility to get internal low-level information, the most important being info about the superblock:
btrfs inspect-internal dump-super /dev/sdb1
and about the chunk tree:
btrfs inspect-internal tree-stats /dev/sdb1

Deduplication

While in-band deduplication is not yet available, offline dedup is.
It is not provided by btrfs-utils, but you need a third party tool from AUR, e.g. duperemove or bedup.
Deduplication in a broad sense simply means removal of redundant data, which can be especially useful on COW filesystems.
For the moment being, don't worry about it.

---

---

Personal note:
I hope that Calamares gets more btrfs related options in the future.
With Architect you can already use more of the potential of btrfs, like creating subvolumes and setting mount options.

[TomZ]

Great write up! Snapshots have been lifesavers for me not only once.
As far as btrfs receive/send are concerned, I use them a lot as they turn snapshots into backups.
This will even work with incremental backups which I like in particular as I can store lots of backups without the corresponding space requirements.

There is a great thread on btrfs over at the Kubuntu forum which lured me into btrfs and which I can highly recommend. The best practice advocated there works very well for me on Kubuntu and Manjaro!
https://www.kubuntuforums.net/showthread.php/74093-Beginner-s-guide-to-BTRFS

Edit: Encryption works fine for me via an encrypted luks container with btrfs inside

[anon23612428]

Thank you very much!

I will add more info about send/receive at a later point. Or very soon because I'm very interested in that feature. Just have to test it a bit.

[nrde]

Thanks for the well-written quick guide!
I would be interested in how btrfs strikes in everyday life, if you use the partions for years.
How often do you have to maintain your btrfs filesystem?
Does it just go by the way or do you take care of it regularly?

[anon23612428]

Currently all is going well since I installed this system in Q1-2017.
I don't do any maintenance, except for a balance&scrub every month or so.
Up to now, I've been lucky, it survived a few power outages and hard crashes.

@TomZ, unfortunately I don't have time to investigate send/receive, would you mind writing a few lines about it? I could then just add it.

[TomZ]

(My) Btrfs best practice

I promised to write a bit on send/receive but as I started I figured I may as well add a bit more detail.

But before I start I'd like to call out that apart from using btrfs on all of machines for quite a while all of my knowledge is based on the Kubuntu Forum thread I mentioned a bit higher up.

The below is how I use btrfs on my machines with some context so you know why I do what I do.

Some context
The thing which confused me the most with btrfs was that I couldn't figure out how these odd @ and @home folders/subvolumes/whatever which everyone seems to be talking about related to my traditional understanding of hierarchical file systems. Where do they come from, what are they, etc.?

Well, the important point to understand is that like any other hierarchical file system btrfs has got a root! But what you see when you do a ls / running btrfs isn't the real root of your file system but rather a subvolume called @, created during installation of the OS.

During installation the OS installer automatically creates two subvolumes (one for "/" called @, and one for /home, called @home) and mounts these two subvolumes as / and /home via fstab.

Post installation your fstab will look like similar to below, please note the subvol=@ parameter which tells the OS to not mount the uuid but rather the subvolumes inside this uuid with the names @, or @home.

/dev/e74269dd-8612-4b09-b59b-3145959fc5bd / btrfs subvol=@,defaults,ssd,noatime,space_cache,autodefrag 0 1

/dev/e74269dd-8612-4b09-b59b-3145959fc5bd /home btrfs subvol=@home,defaults,ssd,noatime,space_cache,autodefrag 0 2

The consequence is that you will never get to see and to work with the real root of btrfs but only the respective subvolumes!

Complicated? Not really, and it comes with significant benefits!

To bring the benefits of btrfs to live I mount the real root of btrfs to /mnt. Real root means I mount the partition uuid instead of the snapshot of "/" created during installation.
This means you move yourself a level higher to what you get with ls /
I do this again through fstab, but this time without the subvol parameter:

/dev/e74269dd-8612-4b09-b59b-3145959fc5bd /mnt btrfs defaults,ssd,noatime 0 2

When you now run ls /mnt you should see this:

~$ ls /mnt
@ @home

And when you dig into e.g. @ via ls /mnt/@ you will find that this the same as what you get with ls /. Through /mnt/@ you are basically looking at your root from outside in. Same with @home

Now, the cool thing about this is that you can now very easily create a snapshot of your / and /home with these two simple commands:

sudo btrfs subvolume snapshot -r /mnt/@ /mnt/snapshots/@$(date +%Y-%m-%d)
sudo btrfs subvolume snapshot -r /mnt/@home /mnt/snapshots/@home$(date +%Y-%m-%d)

You can call your snapshots whatever you want and put them wherever you want, but I use the name of the source subvolume (@, @home) and add the current date as reference which is this $(date +%Y-%m-%d) variable. I also created a new folder "snapshots" in /mnt to keep things clean.

Your file system will now look like this:

/
/@
/@home
/snapshots

Why would you want to do this?

Here's my prime scenario:

1 GB of new OS updates, cool! Let's install and cross fingers...
Or...run the two commands above to take a snapshot of your OS and run pacman -S
...which will take 1 second.
If things break you simply go to /mnt, mv @ to let's say @old, mv the snapshot you have just taken to @, same for @home, reboot and you are back in business.

Full backup and restore of / and /home in less than 5 minutes!

Snapshots are usually read only which means that the full set of commands for a complete system restore looks like this:

#Move the broken stuff aside
mv /mnt/@ /mnt/@old
mv /mnt/@home /mnt/@homeold

#Make the snapshot rw
btrfs property set -ts /mnt/snapshots/@_yourname ro false
btrfs property set -ts /mnt/snapshots/@home_yourname ro false

#Put the snapshot in place of the current OS
mv snapshots/@_yoursnapshot /mnt/@
mv snapshots/@home_yoursnapshot /mnt/@home
reboot now

Important note: from my experience it’s only desktop OS installers which will automatically create the @ and @home subvolumes. I know Kubuntu and Manjaro do.
My Ubuntu server did not, which confused the hell out of me until I finally understood what was going on. If @ and @home are not created during installation you can create them manually but this is not as straight forward as it may sound, you can read up on my learning curve in the Kubuntu thread mentioned earlier

Now on to
btrfs send/receive

Snapshots are live safers in many cases, but they are no backups because a snapshot always resides inside the btrfs tree it originated from. If the disk dies, your snapshots die!

Hence we need a way to send the snapshots to a different btrfs file system, e.g. a USB attached backup drive

Syntax is pretty simple again

sudo btrfs send /mnt/snapshot/@_2019-01-01 | sudo btrfs receive /media/Backup

One of the cool benefits of this approach is that the backup is fully accessible to you. Means if you ever need just one file from the backup you can browse through it and copy files like with any any other file structure, it's not a big binary blob which requires a dedicated backup tool to restore content.

Please note that this command will only work if the destination is directly attached to your machine and the destination drive is btrfs formated!

In case of a filesystem different ot btrfs or a backup over the network you can still use btrfs send but in this case you need to pipe the output to a file.
E.g.

sudo btrfs send /mnt/snapshot/@_2019-01-01 > /media/backup/MyBackupFile.txt

I do run backups over a SMB share but stopped using btrfs send > file for this as the outcome is a binary blob which needs to be fully sent back via btrfs receive before it's content can be accessed which I don't like. I use good old rsync for this.

Another cool side of btrfs send is that it can also create incremental backups. Means that only the few files which changed between snapshot1 and snapshot2 will be sent with dramatic impact on storage and time.

For this to work the parent snapshot needs to be available at both ends, source and destination, so in case you ever need to restore the snapshot from your backup drive all files are available.

Here’s an example using only @, in real live you have to do the same for @home to keep them in sync:

#Day 1: Snapshot
sudo btrfs subvolume snapshot -r /mnt/@ /mnt/snapshots/@_day1
#Day 1: Backup
sudo btrfs send /mnt/snapshot/@_day1| sudo btrfs receive /media/Backup

#Day 2: Snapshot
sudo btrfs subvolume snapshot -r /mnt/@ /mnt/snapshots/@_day2
#Day 2: Backup
sudo btrfs send -p /mnt/snapshot/@_day1 /mnt/snapshot/@_day2 | sudo btrfs receive /media/Backup

Explanation:
Day1 we will send a regular backup, on the second day the day1 snapshot will already exist on the backup drive so we can tell btrfs to only send the difference between day2 and day1. -p stands for parent snapshot.

In case your drive died and you need to restore it wouldn't matter that day2 was created through an incremental backup, all that’s required is

sudo btrfs send /media/Backup/day2 | sudo btrfs receive /mnt/@_day2

to get our data back. Now you would run through the renaming exercize as outlined above:

mv @ @old, mv @_day2 @, reboot and you are back in business!

To close things off here’s a simplified version of my backup routine I use on a system with 2 internal drives. Again only showing @ to keep things simple, you will have to do the exact same for @home:

#Snapshots
snap="/mnt/snapshots/@_$(date +%Y-%m-%d)"
if [ ! -d $snap ]; then
echo"I: Creating $snap"
btrfs su snapshot -r /mnt/@ $snap
echo "I: Snapshot created!"

else
echo "W: Updating $snap!"
btrfs subvolume delete $snap
btrfs su snapshot -r /mnt/@ $snap
echo "I: Snapshot updated!"
fi

#Backup to second drive
snap="/media/backup/@_$(date +%Y-%m-%d)"
if [ ! -d $snap ]; then
echo "Send $snap"

else
echo "W: Updating $snap!"
btrfs subvolume delete $snap >> $log
fi

files=(/mnt/snapshots/@)
ii=${#files[]}
btrfs send -p ${files[$ii-2]} /mnt/snapshots/@$(date +%Y-%m-%d) | btrfs receive /media / backup

Number of snapshots
The last thing I want to mention is that apparently one should keep the number of active snapshots below 10 or so as otherwise the OS will have to do quite a bit of traversing through snapshots to find the required file. I have never tested what happens when I go higher because the rational behind the limitation makes sense to me. Part of the script above is keeping the number of snapshots to 5 which I can post as well if anyone is interested.

Obviously this restriction doesn't matter for backups, you can have as many as your drive can hold, and if you do incremental ones with little changes between the individual backups, your drive will hold lots of backups.

btrfs and encryption
Unfortunately btrfs does not have built in encryption. But an easy way around this is to create a luks partition and run btrfs inside the encrypted container. You will lose some of the btrfs benefits related to handling partitions but hey what you get is still better than what e.g. ext4 offers.
The Manjaro installer allows you to create an encrypted btrfs partition during installation which means there is no extra work required.

A sample /etc/fstab for a a machine with 3 partitions on the boot drive and a separate data drive(s) mounted to /media/data will look like this:

UUID=4D62-1DEA /boot/efi vfat defaults,noatime 0 2
/dev/mapper/luks-e74269dd-8612-4b09-b59b-3145959fc5bd / btrfs subvol=@,defaults,ssd,noatime,space_cache,autodefrag 0 1
/dev/mapper/luks-e74269dd-8612-4b09-b59b-3145959fc5bd /home btrfs subvol=@home,defaults,ssd,noatime,space_cache,autodefrag 0 2
UUID=3511221b-0a77-40c6-ae4e-dd0b8f5d0778 swap swap defaults,noatime 0 2
/dev/mapper/luks-e74269dd-8612-4b09-b59b-3145959fc5bd /mnt btrfs defaults,ssd,noatime 0 2
/dev/mapper/luks-5126b74d-cf6a-4213-be6c-9413db149187 /media/data btrfs defaults

Quite a long write up but I hope this all makes sense...

[anon23612428]

Awesome, thanks for the write-up!
This thread has now become a really good resource for anything related to btrfs!

[TomZ]

@anon23612428

Create an unformatted partition on the new drive, let’s assume it’s /dev/sdb1 .

You can even work completely partition-less:
btrfs device add /dev/sdb /home

@anon23612428

In some cases it is useful to disable COW, for example for partitions that hold VirtualBox VDIs.
This can be achieved with chattr +C name-of-file (or name-of-directory).

Indeed! lsattr allows you to check the settings.
E.g. lsattr ./VirtualBox\ VMs/

[anon23612428]

@TomZ

You can even work completely partition-less:

True, I didn't mention it explicitly because partition-less setup was already mentioned in "filesystem creation".

[TomZ]

One thing I find important when it comes to btrfs and partitions is that the need for dedicated / and /home partitions goes away because the segregation of the two is done via the @ and @home subvolumes created during installation.
This way you get the benefits of dedicated partitions for / and /home

• you can back them up separately
• send them to other drives individually
• even have multiple OSs on the same btrfs filesystem all using the the same /home

...without the downsides of fixed size partitions

• Need to decide on partition sizes during installation
• Space isn't shared between / and /home
• Partition resizing required if needs change

The other important bit to know about partitions is that if swap is required it can't be a swap file! Btrfs does not yet support swapfile which means a separate swap partition is required.

[anon23612428]

Yes I agree, problem is that Calamares doesn't yet fully support subvolumes. While it does create a subvolume in each partition, it does not create multiple subvolume in a single partition.
That's also why my current setup, installed in 2017, has two separate partitions for / and /home, and I'm too lazy to reinstall this well-oiled machine.

[TomZ]

Maybe this has changed since 2017?
Calamares supporting btrfs + encryption was what lured me over from Kubuntu and I got my 2 subvolumes (@ and @home) in one partition via the installation.

[anon23612428]

Yes, that's very much possible. Since I will have to reinstall my laptop soon, I will try again.

[openminded]

Since btrfs now supports swap files (kernel 5.0 and newer) I recently tried to make a fresh Manjaro install with M-A onto another partition and set up a swapfile on it. With some reading of Arch Wiki I ended up with the following swap-related entries in fstab:

# swap
UUID=acaa7834-07af-4274-a584-f449dcd51fc0	/swap     	btrfs     	rw,noatime,compress=no,ssd,space_cache,commit=120,subvol=@swap	0 0
/swap/swapfile none swap defaults 0 0

Basically I just took a bunch of options from home subvolume entry and modified it a little (compress=no, subvol=@swap). And it seems to work this way. However I am not an expert on btrfs in any way, that's why I'm wondering what do you guys think of it?

[TomZ]

I looked into changing to a swap file but in the end went for in memory zfs.
It just got more complicated the more I read up on it.

1. To be able to hibernate I (think) need at least my amount of RAM, some say even twice the amount, so here go 32GB of precious SSD storage, nothing gained space wise.
2. Then I read somewhere that the swapfile should be on it's own subvolume and CoW switched off.
   Which is when I felt this to be too much plumping just for swap and I went with compressed RAM as described in another thread here.

[anon23612428]

@TomZ

I went with compressed RAM as described in another thread here.

Me too
zram that is.

openminded, I think you did it the correct way, just make sure you disable COW, e.g.
remove the swapfile, then
chattr +C /swap
and recreate it again.

[openminded]

OK swap file works fine except for one thing:

systemctl hibernate does not hibernate on swapfile on Btrfs after kernel 5.0

[openminded]

This got patched and merged to systemd master a couple of days ago. Great. Now it is possible to hibernate to and resume from a swap file. I think I gonna migrate to btrfs soon.
The only thing left for me is to arrange a plan on this cuz I don't want to do clean install and move configs and stuff...