How to use USB sticks?

Hello,

I have since many years problems with using usb stick on manjaro.
If I copy files (few hundred Megabyte) to the usb stick it finish normally, but when I want to eject it takes five/ten to (maybe) infinity minutes after that I can unplug my usb stick. During this time the system shows the notification that the usb stick is still in use.
If I unplug it earlier I get data loss.
I bought new usb sticks, using different usb ports (front/ backside) and different devices (desktop/ notebook), even I have installed a fresh manjaro once, but still using usb stick doesn't work really.

I am using fat32 or ntfs on the usb stick in order to have them plug and play.

My question is: how to use usb stick as plug-and-play device?

I know all this sound exaggerated, but using (my own) usb sticks on manjaro is for me like Russian roulette. On my old manjaro installation I had to read so many things to get fairly usb sticks running, but I thought it should run in the mean time out of the box.

The best way to have consistent results is to add a record for USB sticks to /etc/fstab.

(Note: The example I'm about to post here-below applies only for plugging in one USB stick at the time, so it won't work for plugging in multiple USB sticks in different USB ports.)

The record could look something like...

/dev/sdb1 /run/media auto noauto,nofail,defaults,uid=1000,gid=1000,sync 0 0

Change /dev/sdb1 to the correct device special file for the USB stick ─ see the command below this paragraph ─ and /run/media for whatever other mountpoint you desire.

The command that will tell you the correct block device for the USB stick is this here, to be entered in a terminal window after inserting the USB stick...: :arrow_down:

lsblk

Important however is the sync mount option. The default is always async, which means that data written to the device will be buffered and kept in memory until the system is under the least load, and then it will be synchronized with the physical storage medium. sync tells it to synchronize with the storage medium immediately.

However ─ and this is important too ─ there are several things which determine the transfer speed and how quickly a write operation to the USB stick can terminate. The first of them is the USB transfer speed of the stick. There are significant differences in speed between the different USB standards ─ USB 1, USB 2 or USB 3. So it all depends on the speed of the USB port and the speed of the USB stick itself.

The second important factor is that USB sticks use flash memory, but do not support the TRIM command, nor do they have any wear-leveling logic built-in. This means that over time, they tend to wear out as you write more data to the device, and perhaps overwrite existing data.

Flash memory cannot be overwritten, so it is actually an operation in which the new data of the file that is to be overwritten gets written to a new location first, and then it must erase all cells that held the original data. And there's only a limited number of times you can do this.

A USB stick that has already been used many times before for writing data, "overwriting" data ─ i.e. copying and erasing ─ and that also is more than half full will begin to show significant performance degradation. And there is nothing that the operating system on your computer can do about that. It's inherent to the design of USB sticks.

SSDs usually also employ flash memory, but they support the TRIM command, and they commonly also employ wear-leveling techniques, in addition to a significant overprovision of the available storage space ─ or otherwise put, SSDs commonly have a significantly higher amount of storage capacity than what they are sold as, exactly so that they would be able to level the wear caused by write-and-erase cycles and write amplification. But that is of course why SSDs cost a lot more than USB sticks.

You can extend the life of a used USB stick somewhat by reformatting it and starting anew. It levels the wear somewhat, as opposed to when you keep on "overwriting" files on it with newer versions of the data.

Hope this helps.

4 Likes

Probably the system is actually still writing (from cache) to the usb stick. This can take quite some time depending on the usb and/or stick speed, amount of data copied and cache size. If the used usb stick has an activity led, you should still see it flickering.
Take at look at vm.dirty_ratio and vm.dirty_background_ratio to adjust for your needs, mentioned here for example:

4 Likes

Is it that same accross different file managers? Have you tried a different DE?

Linux and other systems uses memory to hold data while copying to another device.

Depending on the receiving device type and speed it will take some time.

It is physically impossible to have data flushed to a device instantly - which is why you must use a safe eject from your file manager or execute sync in a terminal.

If you fail to do so - you already know - the data in transfer will be corrupted.

There is no way you can avoid the waiting process before disconnecting the device.

2 Likes

Hi again, I would like to know if a usb stick could be used for ext4 and fat 32 on the drive to use on windows or linux ?

Al get two sticks but I just wondered bc yous are the geniuses who make it all happen. :slight_smile: thanks.

Don't hijack threads, use the search function and/or open your own help topics.
Windows can't read ext4 so if you need a Windows pc reading this usb stick you'd have to use fat32 or exFAT.

2 Likes

thanks for that also not used to any of this sorry am still finding my way round, thanks again.

TL;DR
What you see is absolutely normal as everyone else has already explained.

But there is a solution: If speed is important to you, buy a fast USB stick:
All USB sticks are equal, but some USB sticks are more equal than others! :wink:

The key to look out for is SLC (Single Level Cell) and eMLC (enterprise Multi Level Cell) when buying.

The long version:
As multiple people have already explained, the copy process copies to (unused) RAM and then flushes out that RAM to the USB stick as fast as the USB stick can handle it and that is where you run into your issue.

The technology behind NAND is based on Quantum Physics¹ and these types exist at the time of this writing:

  • SLC stores one bit per cell and is about twice as fast as MLC
  • eMLC is a hybrid between SLC and MLC: a smalll SLC cache and a larger MLC storage medium that allows the writes to continue after your remove it from the PC and when you re-insert it to another PC.
  • MLC or Multi-Level Cell stores 2 bits per cell and is about 50% faster than TLC.
  • TLC or Triple-Level Cell stores 3 bits per cell and is about 33% faster than QLC
  • QLC or Quad-Level Cell is the cheapest of them all, but also the slowest...

So:

  • if you care about speed only, buy SLC.
  • If you want a good balance between price and performance, buy eMLC
  • If you want to have good quality and long life but not break the bank, buy MLC
  • If you want OK quality, buy TLC.
  • If you only care about price, buy Cheap! Crap! Inc! ™ and be patient while it flushes the data to the stick!

Note 1: Yes, you're using Quantum Physics in every day life. :innocent:
Note 2: SLC is about 4 times as expensive as QLC, but also has a life-time that is 4 times as long as QLC!
Note 3: I still have a Kingston 8GB SLC USB stick on my keychain that cost me 40€ more than 10 years ago, but it's still working whereas all the other Cheap! Crap! Inc! ™ USB sticks have been thrown into the garbage a long time ago...

4 Likes

Thanks, for your responses. In midweek I'll have time to read, test and respond.

1 Like

I am using xfce and as file manager thunar.
I have tried a few usb sticks, cheap and expensive ones, but what kind of transistor technology they use I don't know.
The sync option sounds interesting, but I am not sure if I want a permanent /etc/fstab entry for removable devices. However, I wasn't aware that I can set async/sync in the /etc/fstab that's helpful.
I tried dirty_ratio, dirty_background_ settings and, I think will read more about that and then do some tests with my system.

Now you know what to look for: when buying expensive, verify whether they use the right technology...

sync will not allow you to eject the USB sticks faster, it will just sit there for the same amount of time as an eject.

The only way to make them faster is to buy another technology as it's a hardware limitation... (same as SSD and HDD: you'll never get a HDD to be as fast as an SSD)

:innocent:
¯\_(ツ)_/¯

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Forum kindly sponsored by