Regarding default file descriptor limit

I tried to send this via the Feedback form, but nothing happened after pressing "Send email".

I am pretty sure you have already heard news of Valve's Steam Play, which uses modified WINE to play Windows games on Linux. I've tried it personally with pretty good success.

More information in links below:

The third link mentions limit of file descriptors. This I believe defaults to 4096 on Manjaro. I first encountered this limit when I attempted to build "code" AUR-package. I had to increase the limit in order to successfully build the package.

The esync-link further reinforces the fact that the default Linux limit of 4096 is way too small. As such, I would like to propose to change the default limit so that people wouldn't have to do it manually.

Seeing as many (all?) Manjaro spins already bundle Steam by default, I believe this would be in the best interest of Manjaro.

How is that increased?
If it's in a conf file somewhere, please post the path to it. :slight_smile:

EDIT: Oh, and more importantly, are there any downsides to increasing it?

It's in the last link he posted:
"/etc/security/limits.conf"
Don't ask me what to do with it :neutral_face:

Edit: Oh, it's explained in there too

"limits.conf" is the correct file if your system does not have systemd. Otherwise the correct files are "/etc/systemd/system.conf" and "/etc/systemd/user.conf". At least I am not aware of any Manjaro spins that do not have systemd.

And what you need to do in system.conf and user.conf, is to uncomment "DefaultLimitNOFILE" and put a value in it, such as "DefaultLimitNOFILE=1048576".

There is also an Arch wiki article about this. https://wiki.archlinux.org/index.php/Limits.conf

So what should be the limit? What does Valve recommend?

One would have to look into SteamOS to find what Valve officially uses after Steam Play is out of beta. But the number that the esync developer recommends is 1048576, which I believe is intentionally ludicrously high so that the limit is not reached when playing games.

I don't think the number matters. The limit has to be high enough not to be reached when playing games such as Skyrim when heavily modded. I am sure there could be games that require even more, if not today, then eventually.

Now I'm wondering once Valve gets this working right if they will launch a Steam Machine 2 later? I thought one from Zotac was nice.

This is already in discussion upstream at Arch, https://lists.archlinux.org/pipermail/arch-dev-public/2018-August/029335.html

If they implement that then Manjaro will immediately benefit too. I don't see any downsides of overriding it ourselves in the meantime.

I have to ask but will increasing the limit also increase memory requirements as well for some games?

Sorry, but that does not seem to be the same thing as what is being discussed here.

Well, think about it like this. If a game would like to use more file descriptors than system is willing to give, the game would most likely crash or exhibit undefined behaviour. So at that point increased memory usage is kind of irrelevant.

Yes but I have 16GB now. This should be enough right?

Just go by what the game's system requirements say. The game is going to attempt to use X number of file descriptors regardless of what your current limit is set to. If it fails to do so, it'll most likely crash.

The same happens when you're attempting to compile a software from source, and the compilation requires simultaneous access to more files than what your limit is set to, the compilation fails. This happens if you attempt to install the code (Visual Studio Code from Microsoft, the github version) package from AUR for example.

In the end, it does not really matter if your game crashes because it uses more memory than it can, or if it crashes because of file descriptor limit being reached. Of course with swap file or partition, OS memory management can move stuff from RAM to swap if it comes to it, but you're not gonna get extra file descriptors out of the fly.

OK, so can you please come up with a PoC implementation with a list of potential benefits and drawbacks. Then this can be made into a proper #manjaro-development:feature-request .

Do you have a template I can use? Should I post the PoC here, or?

The systemd package is here:

You can e.g. post a diff.

I have no idea where values for system.conf and user.conf are. You'll have to do without a diff I guess.

/etc/systemd/system.conf
Before: #DefaultLimitNOFILE=
After: DefaultLimitNOFILE=1048576

/etc/systemd/user.conf
Before: #DefaultLimitNOFILE=
After: DefaultLimitNOFILE=1048576

Benefits: Building packages that use large number of file descriptors no longer fail, for example the code package in AUR. Games that use large number of file descriptors won't crash or have undefined behavior. See https://github.com/zfigura/wine/blob/esync/README.esync for more details.

Drawbacks: Not aware of any.

Ubuntu have set nofile 1048676 (2^20) but I can't yet see where it is being set.

The 4096 hard limit is set in the kernel source, https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0ac1ee0bfec2a4ad118f907ce586d0dfd8db7641 . It was set seven years ago (up from 1024) so it might be something for the kernel developers to look at again rather than have each distro "solve" this is a different way.

An easier way to accomplish this change in the short-term may be to create a new file /etc/systemd/system.conf.d/limits.conf with e.g.

[Manager]
DefaultLimitNOFILE=1024:1048576

This change needs to be tested more widely.


I'd suggest that a change to this default should be raised upstream, whether systemd or the kernel, so it can be "fixed" for everyone. @KeeperB5 please file this request with the upstream projects.

Apparently this has been implemented now, it will all work perfectly without needing any testing, so noone needs to do anything more and you can all just ignore me.

Forum kindly sponsored by