Manjaro debates: How to avoid Stable updates breaking the system?

Maybe we should discuss branches and update forms. For example testing and unstable branches get updates but this updates not like stable they are pieced IMHO testing branch updates should be previous stable updates because we could see whole process to what cause the problem.
And we cant judge people and we can't blaim for they have lack of knowledge because Manjaro claims that "We are an end user friendly distro"
Best regards...

3 Likes

This is NOT a user-friendly update procedure. Beginners will almost fail and worry about this update process you recommended. How do i know if i use systemd at all?

I would recommend to wait until the systemd issue get fixed, because this is not a stable update!

2 Likes

Seriously?
In this day and age, if you havet taken steps to specifically avoid systemd you can assume you are running it.

FOR SYSTEMD USERS IT IS RECOMMENDED TO USE PACMAN AND TTY

I found it an odd thing to warn about, and only warn systemd users. The advice would work for all users, so why make the distinction.

But posting that in a release notice when virtually all of us are running automatic update checking package managers (especially novices) is going to be way way too late.

I think it needed more time in the testing repo before it got pushed to stable, but if any update is going to have a damaging effect, there should be a way to push a small file to all the mirrors telling people to check the forum before installing.

Pamac worked just fine for me.

I don't think so. The problem is another, I think. I'm using the unstable branch and update every day. So I see only the problems, which single packages causes. Similiar in the testing branch, there are many updates a week. Only in the stable branch come all updates together. And so the problems with the systemd update could cause that the mkinitcpio for the 4.8.1 was missing and the start with this kernel ends with kernel panic.

And this time, it was a special situation too, because the kernel 4.8 was causing so much trouble since rc9. And the focus in testing was there.

I think, we should aware, that the stable update needs a test for one or two days two. It's not such a nice idea to execute the update on systems, which are working productive. On this systems it seems better to wait three or five days until existing problems are explored and solved.

4 Likes

Well, it is simple. 90% are using SystemD and others use OpenRC which are not affected by this. Also I've updated the instructions to updated systemd first.

quite simple: pacman -Q systemd - if you get an output you have it :wink:

Yeah, success. What I wanted. Wack most of the users with one simple update :wink: Are you sure ? This update came with a stable release of Manjaro 16.10-rc1, which has all the scary pre-applied. We can't test all the changes on all systems. On my end all was fine as I updated systemd from revision 1-4 step by step. Only changing from revision 1 directly to 4 creates this issue. Holding back other updates would have had the same effect soon or later.

So, one sec, let us fix it for you completely:

17 Likes

[quote="philm, post:175, topic:10663"]
We try to avoid these issues, but not all can be avoided
[/quote]I believe that.

I am only suggesting that the numbers of helpers/testers, i.e., the number of eyes looking at candidate software, need to increase. I think if an extra weekend is wedged in the 'testing' period more people will be able to participate.

Clearly, Arch needs a longer/deeper buffer before updates are pushed to Manjaro "Stable"; the person on Manjaro Stable branch, at the very least, needs to always have a bootable GUI system. Beyond that, for 'updates', as you put it, "not all can be avoided".

Extending the standard time between updates to get more eyes participating in "Testing" might achieve less than desired results(more stability) but it cannot hurt to try it, can it?

...at least 2-cents, :slight_smile:

...

[quote="jsamyth, post:182, topic:10663"]
So if you think sddm might be at fault, just try lightdm for a bit. It
[/quote]No, no, not SDDM per se. Its theming is QML based so if QML is whacked somehow( :confused: ), SDDM will act-up.
Yes both sddm & lightdm can be installed(and others too ithink).
I was going to try LDM before I found the QML message but stopped because display-manager.service remained a symlink to sddm.service even after sddm.service was stopped. I did not want to disable it ...but prbly will now & at least try lightdm w/ the kde-greeter for it.

Thanks for the information.

c-ya

In the far-distant (in Manjaro-years) past, updates were batched fewer and farther in between. Anecdotally speaking, the longer the time period between the batched updates, the more frequent the breakage.

This last update batch reminds me of those times, but this was a particularly difficult update. I believe @philm & Co. did as good a job as possible.

To all: I run vanilla Arch on my main machine, one form of Manjaro or another on my alternate machine. In Arch I have updates every day. Every day! If I don't keep up with them, I risk breakage. But if I blindly update, I risk breakage.

What to do? Read. Read here. Read your Reddit groups. Read the Arch forums. Other forums (Spatry, for instance). Join a few mailing lists....

Leading/bleeding-edge means a willingness to risk an occasional fall. If you are not willing, then don't take the risk. :wink:

Regards

14 Likes

This is a question of process fine tuning.
How big is the effect of "more eyes participating" of you increase the time interval vs. the effect of more packages to be updated at once having interconnected risks of breakage.
Also consider another effect of larger updates: more packages to update -> higher chance that some setup (hardware+OS) breaks due to different issues -> support has to bear in mind several different issues at once -> support getting exhausted.

So essentially what went wrong is that there is a problem with systemd when you update "stable" to the new "stable". Testers with "testing" did not experience any problems because their gradual updates of systemd did not expose this problem, right?

So what about creating a new group like "stable-update-testing" or "rc-testing" that get updates exactly the same like intended for "stable", but 1-3 days in advance?

3 Likes

I think like as it has already been said. it won't change much it will just delay the updates (and that they aggreed to report issue of course. ).. the best is to have more courageous user on testing and unstable branches. if there is not enough user in testing and unstable to add a "branche" won't solve much problems on stable. yes to have more package to upgrade increase the risk but not as much as to have more user in testing and unstable raise the different hardware and installation to find out the problems faster. on testing and unstable the systemd update just reported to lose X and that to change to tty and go back to X was enough.

1 Like

Well, I have one system on testing and I did not experience the bug. With my other system on stable, I did experience the bug (although I just had to switch (back) to tty 7, see that the update was finished and I could reboot normally).
In this case, more users in testing would not help anything.

Maybe testing<->stable should not be lengthened by 3 days, but testing<->rc-testing could be shortened by 3 days, so rc-testing is parallel to the last 3 days of testing. Only downfall would be that updates (and what packages are in) have to be set fixed at least 3 days before stable release.

Anyway, I wouldn't set my stable system to testing (need more stability there, don't want "daily" updates and bigger chance that at least one system is working in case of a problem).
But I would be willing to set it to rc-testing: normally same stability as stable unless a problem like the current arises, same amount of updates as on stable and still bigger chance that at least testing or rc-testing is working.
So rc-testing would be for those that prefer stable, but in case of problems have knowledge/time enough to report in forum and see how to proceed further.
Even Forum Announcement for rc-testing could be the rc for stable annoucement (simple copy-paste) + notes/problems/how-to found during those 1-3 days.

1 Like

[quote="c00ter, post:198, topic:10663"]Leading/bleeding-edge means a willingness to risk an occasional fall. If you are not willing, then don't take the risk. :wink:[/quote]Oh Horsesh~t. What part of "Stable" is not understood?
By definition, "When something is stable, it's fixed and steady. "
This so-called "Stable" update was neither fixed nor steady; neither was the previous to the last ...

If I, or anyone, wanted unstable from Manjaro then we would run "Testing" or "Unstable"/Arch.
...

I believe that Phil and other Manjaro dev's do what they can to insure every update/release is stable. Among other reasons, nobody in their right mind would desire over 200 posts, mostly complaints, about an update.
What you butted into was a suggestion to Phil and Manjaro dev's (by way of Phil) as a way to avoid having these kinds of issues in the future.
... And your response is unverified BS about the old days when Manjaro had even fewer users and that stupid "love it or leave it" BS.
If the post cannot be constructive, keep the fingers still.

but if we play with words. as arch have a testing repos. we can say that the normal arch is implicitly called "stable" arch no? :wink:
not that I dsagree with all what you said. and which issues could stay at a minimum or even none if possible on manjaro stable.

2 Likes

[quote="eugen-b, post:203, topic:10663"]This is a question of process fine tuning.[/quote]Yes. I suppose I am trying to get the point across that "Stable", "Testing" and "Unstable" are S2B distinct and that "Stable" needs to be absolutely rock-solid-stable, at least as far as boot to GUi is concerned. The only way I know to achieve that in a FOSS model is by increasing participation "eyes" to catch the critical system bugs.

[quote="eugen-b, post:203, topic:10663"]How big is the effect of "more eyes participating"
[/quote]There is no definitive answer for that in general but it is the core of FOSS and has been working for more years than Manjaro(or Arch, FTM) has been around. It is well proven to work. ...

[quote="eugen-b, post:203, topic:10663"]Also consider another effect of larger updates
[/quote]Why? There is no law that says everything in Testing must go into Stable nor that everything in Testing must remain in testing and not go back to Unstable. If it is too buggy or breaks systems, take it out or, at the very least, do not push it to Stable.
[ I think the only exception to that would be security updates. They must be pushed to $USER ASAP, without system instability as a result. ]

[...]
AFAIK, the only way to identify issues is extensive testing in a lab with 100's of different systems & configurations or get the $USER to test and report.
An extra weekend, when most people have some free time, might get more participants and :. bugs found. If dev's do not have the time to fix and $USER is not offering the required diff, then delay that SW update.

...idunno. The suggestion to try and get more eyes to catch critical issues and fix them before a "stable" release is only just that: a suggestion.
The model being used, Arch=> Unstable => Testing => Stable apparently demands more eyes if it is to produce a "stable" Manjaro update/release. Maybe y'all can resolve the issue by renaming the "Stable" branch to something else. Perhaps name it "Rolling" or anything that does not mean "stable", and make the point releases, like the upcoming 16.10, the only "stable" releases.

...

but if we play with words. as arch have a testing repos. we can say that the normal arch is implicitly called "stable" arch no? :wink:
[/quote]idunno, I suppose it is. If I understand the point, Manjaro Unstable is actually S2B "stable" so zipping thru Unstable to Testing and then to Stable should be OK. Right?
But the obvious fact is that assumption is false.[quote="scachemaille, post:214, topic:10663"]
not that I dsagree with all what you said. and which issues could stay at a minimum or even none if possible on manjaro stable.[/quote]I don't believe there are ever zero bugs in complex systems, only the ones that have not been found. ...hence security and bug fix updates.
Of course the ideal is zero but my suggestion is not about that. It's only addressing the lack of stability and how to fix that without killing dev's from overload or buying 100+ systems to use in a lab as testbed.

Unlike MS or Apple OS, Linux OS, DE and SW have always been participatory operating system and supporting software. If $USER does not participate, FOSS model fails.
If there is another/better way I'm all ears, so to speak. :ear:

c-ya

2 Likes

Maybe this could be a good time to suggest splitting to a new topic if there is so much to say?

2 Likes

First, let me say "Thank you!" that you think so much about improving Manjaro!
But there is a law that you sometimes cannot hold off a package in a rolling model for too long.There are dependencies and they need to get updated, because they are dependencies of other packages. Eventually you need to update everything if you start updateing some packages.
I found it out in a practical example of libreoffice. I held it back from updates. Then because of dependencies I needed to hold back evince, then gtk3.

Good idea! Also this can be merged with similar statement in the topic about kernel 4.8. But I'ma bit too sleepy to do it at the moment. :sleepy:

2 Likes

Please do not act in such a threatening manner.

I was relating the consequences of prior batch updating and the fact that the further between batch updates, the more reports of breakage. The closer between batch updates, as has happened in more recent years, the less breakage reported.

That's just a fact, Jack___.

2 Likes

Just some simple minded observations from this simple minded noob.
Manjaro has a small user base viz wizards and gods who still sadly are not PERFECT! Not capitalised . They have not ascended as yet ... this is good for us,,, but they still have to endure SOME members of this COMMUNITY carping. Karma-

Many of these carpers (and most users myself included duh!) have tuned/tricked out there vanilla system thus changing the variable base of how a system may react to some change.

I guess that this time systemd changed the rules that our visual ques were used to.
Many people then exited the update, and screwed it up.
Understandable, but impatient ... I have done the same many times in the past. No one saw it coming.

So...

Analyse,
fix,
Learn and move on.

Hardware varies... DUH!

Lots-0h-verables, interactions, affects, and effects... kind of like herding cats @LizziAS , This time on a three legged pony, go figure?

The longer between updates the more variables of course. Throw in one high level update and hold on! Have fun.

Hind sight is always 20/20! Watch the blame.

Was this or any of the updates worst then say Windows 10? Don't they run something like 20-25% problem rate? That is what a corporate user of windows told me. Manjaro and Linux is not windows hence we tend to be a bit more edgy?! And what else?

As for solutions:

Sure more testers would be a big help. But lets have **

More and perfect testers!

** Yea that's the ticket!

Realistically: Maybe we should all talk up this distro to bring more users in.
Positive helps---->^

...Check it... DistroWatch :stuck_out_tongue_winking_eye: Has had Manjaro in the top bal bla for a long time now. (Love the pic!)

Manjaro is superior because of the vision of philm and the @*dev and its fantastic @*community, non-of which are perfect. So relish the chance to learn from them and contribute to the community as you can,
while we can.
Simply because that is what Manjaro is about.

In any case ,
happy trails... :kissing_heart: community et.al.

I agree on you on this one, but I don't think it applies to a stable branch at all. I used arch in the past and, yes, a long time between updates increases the breakage chances because (besides unseen/unsolved bugs) you often get isolated package updates which can expect a certain version of another package/file previously updated. This happens mostly between major package releases, and, yes, it requires a lot more reading.

In a stable branch you get a complete set of updates which should be verified to work together, so there is less tendency to break the system, independently of the time span between updates. I'm not saying it can't happen, of course it can, but there isn't a linear correlation with a system like vanilla or testing arch. I don't always update my system and I never had a break with Manjaro (I had one, but it had to do with the way the bios dealt with a pci sata extender card, which caused the rebuilding of initram to fail). What I do get is more file conflicts. That is, nothing gets updated until I deal with the conflicts. My Wife's desktop is usually updated only when there is a new Manjaro release (even so, not always!), and I've never updated my Sister's desktop since I set it up in Christmas (don't remember if it was 2014's or 2015's, still running KDE4)!!! I'm going there this year again, so lets seen how it goes...

@neognomic In the end, around 70% of the users had no problems. Manjaro usually has much better results, but I don't think it was that bad. The issue was readily identified and new directions were given (despite the warning being already there from the beginning). Apparently the problem is on restarting the service manager after updating version 231-1. Excellent work, rapid response!

Would a longer time span between stable updates solve these issues? I doubt it!! What I think would help to minimise problems is:

  1. More hardware diversity in testing/unstable branches. This can only be achieved with more users (sorry I can't help right now - I lack the disk space and time needed - my hardware is ■■■■ anyway).
  2. Users with failed updates trying to figure what failed (even if it is by simply downgrading) and posting their results.
  3. PEOPLE READING UPDATE POSTS/ANNOUNCEMENTS BEFORE ACTUALLY UPDATING. This behaviour needs to be learned by any computer user who wants to maintain his system healthy and productive. This statement is valid for ANY system (even Windows - if you have the choice to update or not!), but even more in a rolling Linux distro.

Cheers,

2 Likes

Some of my thoughts:

  1. the bug was fixed but the problem is how it was fixed. In my first post, the first reference is the actual response of Linus about the problem. Basically, in fixing the bug, one of the maintainers used a feature that would just stop Linux.

  2. For former windoze users, releases of the latest and greatest would be buggy. So, most users would wait until the .1 release. Windows 3.0 was a major disaster, they fixed the problems and 3.1 was the winner. As late as Windows 98, it is the second revision that really worked and was a pretty good piece of software.

  3. Software is very complicated and OS software is the most complicated. Such software must work successfully with a variety of CPU's of different ages, but it must also not break older parts while introducing new parts. For windoze, they have only one DE (desktop environment) to deal with. Linux has a lot of DE's, each of which adds they own problems, when using the latest OS.

  4. This can be confusing, as we all know. @LizziAS has been running the latest release with no problems, but @Takeru, @eugen-b and @g33zr, among others have had real problems. @jsamyth mentions that it runs fine for him, in VMWazre but notes that he was not stressing it very much.

Now, some thoughts about what this means now and in the future:

a. Do not expect the latest and greatests software, whether an OS or a specific program to work perfectly from the start. Developers do their best, but 'stuff' still happens.

b. If you want to test the latest, do so on a different machine. Use it as a sandbox so you do not mess up your current settings and work.

c. Learn how to back up your current useful systems and work on a regular basis or at least before trying a new Kernel. I cannot stress this too much.

d. We all have our favorite DE's, I prefer KDE but your choice is just as valid.

e. I still run 4.something, it works that is all I expect of it. I would love to upgrade but I know the trouble that may happen, so I will wait. As soon as @philm releases Plasma 5.8, I will probably take a chance and upgrade to it. It does look so hot. BUT, not until my work and the system has been backed up.

FOSS allows all of us to do what we want. At times, that is very risky. But we can do it. We are all thinkers and pretty good at it. So enjoy all of it.

have fun and sometimes, turn off the computer, and go for a walk...

gary

4 Likes

Forum kindly sponsored by