Bug 363851 - "A stop job is running for Session 1 of user neon" during shutdown
Summary: "A stop job is running for Session 1 of user neon" during shutdown
Status: RESOLVED FIXED
Alias: None
Product: neon
Classification: KDE Neon
Component: Live/Install images (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR minor
Target Milestone: ---
Assignee: Neon Bugs
URL:
Keywords:
Depends on: 364340
Blocks:
  Show dependency treegraph
 
Reported: 2016-06-02 16:33 UTC by Thomas Pfeiffer
Modified: 2016-09-28 10:40 UTC (History)
13 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
attachment-20443-0.html (3.82 KB, text/html)
2016-06-11 17:05 UTC, Paul L.
Details
A stop job is running_1.png (85.24 KB, image/png)
2016-06-11 17:05 UTC, Paul L.
Details
A stop job is running_2.png (85.62 KB, image/png)
2016-06-11 17:05 UTC, Paul L.
Details
attachment-21709-0.html (3.96 KB, text/html)
2016-06-11 17:27 UTC, Paul L.
Details
Processes.png (396.55 KB, image/png)
2016-06-11 17:27 UTC, Paul L.
Details
attachment-25466-0.html (3.73 KB, text/html)
2016-06-11 18:38 UTC, Paul L.
Details
Processes.png (396.55 KB, image/png)
2016-06-11 18:38 UTC, Paul L.
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Pfeiffer 2016-06-02 16:33:22 UTC
Created attachment 99452 [details]
attachment-20443-0.html

After a live CD session, shutdown gut stuck on "A stop job is running for Session 1 of user neon" until timeout.

Reproducible: Always
Comment 1 Jonathan Riddell 2016-06-02 16:35:25 UTC
This is common to distros like Fedora too.  There's a process which is spawned, possibly gpg, and nothing is reponsible to shut it down so the shutdown hangs until a timeout in systemd means it gets killed
Comment 2 Harald Sitter 2016-06-02 16:46:53 UTC
After weeks of pondering I am actually thinking ksmserver needs to run shutdown through sddm. and sddm needs to TERM all of its sessions, so when systemd hits a running session it will actually make sense for systemd to wait (i.e. stuff isn't done dying yet) and eventually kill the stuff (i.e. stuff didn't manage to die on its own).

I do however consistently fail to have anyone who knows anything about this stuff comment on how this is meant to work or not to work. So, I am right now assuming that Linux is not meant to be shut down and when you happen to have it shut down it is because you were lucky and the stars aligned just right. Either that or the whole of ksmserver, sddm, logind, systemd need to come up with a way that a session's processes are actually ended before the relevant systemd session unit is meant to quit.
Comment 3 Harald Sitter 2016-06-02 16:50:49 UTC
Oh yeah. FYI, this also happend with lightdm last I checked. So it might as well be that ksmserver should make sure the session is dead. As I was saying, the documentation, that I didn't manage to find, probably would suggest that every of the involved pieces of software should kill the session, or maybe none, or maybe some.

Also since I just remembered. There is also consolekit. Maybe consolekit should end the session.

#WhatAMess
Comment 4 Louis Moureaux 2016-06-09 16:16:42 UTC
Hello,

This is related to the use of logind. Old session managers used to kill user processes when the session was closed. logind doesn't do that by default, but applications didn't change their behaviour. Starting with systemd 230, the default setting will change to kill processes. In the meantime, just set KillUserProcesses=yes in logind.conf.

See https://github.com/systemd/systemd/blob/master/NEWS#L29

Louis
Comment 5 Martin Steigerwald 2016-06-10 10:55:22 UTC
In Debian there has been a lengthy discussion and bug report about "KillUserProcesses=yes" since it basically breaks screen sessions started by the user and many other things.

Bug#825394: systemd kill background processes after user logs out
https://bugs.debian.org/825394

As a consequence of this Ubuntu/Debian systemd maintainer Martin Pitt reverted this change of default config for now.

I certainly would also disable it, cause in case I need to restart my Plasma session due to a hangup which still happens, while in a screen a borgbackup is running, I of course want that borgbackup to continue its work.

If processes of a user session do not shut down on session shutdown, it is a bug – except for those where it is intended, like a screen session. Just killing all processes just masks that bug and invites to write broken software and makes it necessary to separate a screen or otherwise long running process from the session where it was started from, in this case even probably by using systemd specific API.

Also setting to confirmed, cause, well was confirmed here already by Jonathan.
Comment 6 Harald Sitter 2016-06-10 13:03:42 UTC
In a systemd world I think you are meant to start it as a scope
> systemd-run --scope --user screen
and let systemd manage the thing properly.
Not being able to differentiate between what is actually meant to run "forever" and what is meant to die with the session is exactly why user scoping a persistent background process is meant to solve.
Comment 7 Thomas Pfeiffer 2016-06-10 13:32:54 UTC
Also, in all honesty: Users who need processes to continue running even after logout are _not_ a significant enough portion of neon's target audience to change the _default_ config to better suit their needs.
The vast majority of - even KDE enthusiastic - end users only care about their processes as long as they are logged in. They usually only log out to shut down or reboot, in which case none of their background processes would survive anyway.

Neon is not Debian, the two have quite different target audiences. Those few who need background processes to continue running after logout can still easily change their systemd config. It's not worth to deteriorate the majority's experience just to cater for the fringe.
Comment 8 Martin Steigerwald 2016-06-10 18:42:48 UTC
Entirely your choice and in the end I don´t care, cause I do not intend to use Neon. Yet, you mask bugs this way.
Comment 9 Thomas Pfeiffer 2016-06-10 18:49:29 UTC
(In reply to Martin Steigerwald from comment #8)
> Entirely your choice and in the end I don´t care, cause I do not intend to
> use Neon. Yet, you mask bugs this way.

Sure, I'm all for fixing the misbehaving applications, but "Let's expose other people's fuck-ups so they'll fix them" has not been a very successful approach for us in the past. 
Usually what happens is that those who fucked up don't do anything about it, and our users blame _us_ for the problems.
Comment 10 Paul L. 2016-06-11 17:05:11 UTC
I’m running KDE neon in an Oracle VirtualBox VM. I keep it fully apt-get dist-upgraded.

If I leave #KillUserProcesses=no, then the shutdown process ‘hangs’ once the KDE neon 5.6.90 logo screen appears for a full 1min 30s. I’ve screen-captured this:



Where is this second session of ‘me’ coming from? It’s this second session that is hung (I believe it’s actually dead — a zombie?) and causes systemd to delay the shutdown; it waits for the second session to terminate “on it’s own”, and when, after the default time of 1min 30s, it hasn’t, systemd then kills it and the shutdown finishes as it should.

Paul Loughman
snowhg@icloud.com




> On Jun 10, 2016, at 1:49 PM, Thomas Pfeiffer via KDE Bugzilla <bugzilla_noreply@kde.org> wrote:
> 
> https://bugs.kde.org/show_bug.cgi?id=363851
> 
> --- Comment #9 from Thomas Pfeiffer <thomas.pfeiffer@kde.org> ---
> (In reply to Martin Steigerwald from comment #8)
>> Entirely your choice and in the end I don´t care, cause I do not intend to
>> use Neon. Yet, you mask bugs this way.
> 
> Sure, I'm all for fixing the misbehaving applications, but "Let's expose other
> people's fuck-ups so they'll fix them" has not been a very successful approach
> for us in the past. 
> Usually what happens is that those who fucked up don't do anything about it,
> and our users blame _us_ for the problems.
> 
> -- 
> You are receiving this mail because:
> You are the assignee for the bug.
> _______________________________________________
> neon mailing list
> neon@kde.org
> https://mail.kde.org/mailman/listinfo/neon
Comment 11 Paul L. 2016-06-11 17:05:14 UTC
Created attachment 99453 [details]
A stop job is running_1.png
Comment 12 Paul L. 2016-06-11 17:05:14 UTC
Created attachment 99454 [details]
A stop job is running_2.png
Comment 13 Harald Sitter 2016-06-11 17:10:18 UTC
If you were logged in on a terminal, or you were logged in over ssh, or you switched user, or you logged out and back in again, you would end up with a second session. There's possibly other cases, but generally those are the main things.

Pretty much anything that resembles a login of any type will create a session.

And pretty much any process created as part of login but not killed by some entity other than systemd would then block shutdown.
Comment 14 Paul L. 2016-06-11 17:27:22 UTC
These are the processes running after a starting KDE neon (Oracle VirtualBox VM), logging in (SDDM), and opening a konsole on the Desktop:



/etc/systemd/logind.conf #KillYUserProcesses= is currently set to ‘no’.

When I exit konsole, that process should be killed, so if I then do a shutdown; K Menu > Shut Down (I’m using Application Dashboard), which of these processes is Session 2 for Paul?

Paul Loughman
snowhg@icloud.com




> On Jun 11, 2016, at 12:10 PM, Harald Sitter via KDE Bugzilla <bugzilla_noreply@kde.org> wrote:
> 
> https://bugs.kde.org/show_bug.cgi?id=363851
> 
> Harald Sitter <sitter@kde.org> changed:
> 
>           What    |Removed                     |Added
> ----------------------------------------------------------------------------
>           Assignee|neon@kde.org                |neon-bugs@kde.org
> 
> --- Comment #13 from Harald Sitter <sitter@kde.org> ---
> If you were logged in on a terminal, or you were logged in over ssh, or you
> switched user, or you logged out and back in again, you would end up with a
> second session. There's possibly other cases, but generally those are the main
> things.
> 
> Pretty much anything that resembles a login of any type will create a session.
> 
> And pretty much any process created as part of login but not killed by some
> entity other than systemd would then block shutdown.
> 
> -- 
> You are receiving this mail because:
> You are the assignee for the bug.
> _______________________________________________
> neon mailing list
> neon@kde.org
> https://mail.kde.org/mailman/listinfo/neon
Comment 15 Paul L. 2016-06-11 17:27:24 UTC
Created attachment 99455 [details]
attachment-21709-0.html
Comment 16 Paul L. 2016-06-11 17:27:24 UTC
Created attachment 99456 [details]
Processes.png
Comment 17 Paul L. 2016-06-11 18:38:30 UTC
These are the processes running after a starting KDE neon (Oracle VirtualBox VM), logging in (SDDM), and opening a konsole on the Desktop:



So which one of these is the offending session? Or is it even identified here?

Paul Loughman
snowhg@icloud.com




> On Jun 11, 2016, at 12:10 PM, Harald Sitter via KDE Bugzilla <bugzilla_noreply@kde.org> wrote:
> 
> https://bugs.kde.org/show_bug.cgi?id=363851
> 
> Harald Sitter <sitter@kde.org> changed:
> 
>           What    |Removed                     |Added
> ----------------------------------------------------------------------------
>           Assignee|neon@kde.org                |neon-bugs@kde.org
> 
> --- Comment #13 from Harald Sitter <sitter@kde.org> ---
> If you were logged in on a terminal, or you were logged in over ssh, or you
> switched user, or you logged out and back in again, you would end up with a
> second session. There's possibly other cases, but generally those are the main
> things.
> 
> Pretty much anything that resembles a login of any type will create a session.
> 
> And pretty much any process created as part of login but not killed by some
> entity other than systemd would then block shutdown.
> 
> -- 
> You are receiving this mail because:
> You are the assignee for the bug.
> _______________________________________________
> neon mailing list
> neon@kde.org
> https://mail.kde.org/mailman/listinfo/neon
Comment 18 Paul L. 2016-06-11 18:38:32 UTC
Created attachment 99459 [details]
attachment-25466-0.html
Comment 19 Paul L. 2016-06-11 18:38:32 UTC
Created attachment 99460 [details]
Processes.png
Comment 20 Martin Steigerwald 2016-06-12 08:38:03 UTC
I think one major issue here is that systemd then doesn´t seem to handle shutdown different to log out, i.e. I and quite some other users I read from prefer processes to be kept running on logout, but when I shutdown or reboot the system I prefer it to first SIGTERM and then SIGKILL processes if any are left over. Instead systemd seems to wait on processes on shutdown case, and that requires the KillUserProcess switch be on which then also leads to user processes being killed on logout which is the part that doesn´t make sense to me.

Just as it worked with SysVInit. Sometimes I wonder why its necessary to break stuff that has worked just fine for ages.

The most important question here is always "What do users want?". When I say shutdown I mean it – I don´t mean wait for processes. When I say reboot I mean it – I don´t mean wait for processes. When I say logout, I mean it as well – I don´t mean kill my screen session. Instead of just changing things for the sake of changing them, or only to the view of the world that is correct in systemd developers terms, I wonder how about thinking of a new way to do it and coordinate with any involved parties *before* matter-of-factly breaking existing setups in inventive ways.

So in systemd world either the logout or the shutdown is broken depending on the setting of the KillUserProcesses switch.

Well if its possible to some day have it working out of the box with scopes… but hopefully in a portable way that will work on FreeBSD and other operating systems as well.

*sorry for the rant, I will uncc me from this bug report now, as its better for my mental sanity*
Comment 21 Harald Sitter 2016-06-15 08:44:35 UTC
Has anyone tested this actually?

I just tried and it looks to me that setting killuserprocess is not going to be enough on its own. In particular I seem to recall someone from the Plasma team mentioning that reboot/shutdown do not actually go through logind which is kinda in line with what I am seeing:
- logout -> reboot via sddm = works
- logout -> no lingering session in `logintctl`
- reboot -> stuck 
- shutdown -> stuck

So the way I see it ksmserver also needs to start talking to logind on shutdown/reboot for user sessions to be properly nuked.
Comment 22 Harald Sitter 2016-06-15 09:25:34 UTC
I spoke too soon. Apparently kded just likes to crash every time I want to reboot. Seeing as it then is suspended it wouldn't' react until forcefully murdered.
Comment 23 Harald Sitter 2016-06-15 09:35:33 UTC
https://bugs.kde.org/show_bug.cgi?id=364340
Comment 24 Andreas Hartmetz 2016-06-24 01:15:58 UTC
I'm with Martin. We should make sure that our processes terminate to avoid wasting resources. Independent of that, systemd is wrong to change behavior because the way it used to work was good. Logout - end the session, leave detached processes alone. Shutdown / reboot - kill everything that still lives and only wait for a very short time. It is not hard to understand why that is the right behavior.
Comment 25 Paul L. 2016-07-20 02:36:55 UTC
I've left /etc/systemd/logind.conf #KillYUserProcesses=no

IF I opt to perform a Logout, then once at the SDDM Login Greeter screen (which happens as quickly as one expects it should); then clicking on either the Reboot or Shutdown icon, the selected process proceeds as quickly as one expects it should; the annoying 1 minute 30 second delay does not happen.
Comment 26 romuluspb 2016-07-26 21:19:51 UTC
I suspect it can be responsible by some inconsistencies on multi-screens setups as save session in my experience, has correctly managed my panels position and screen configuration... When it gracefully shut down

Otherwise I get a plethora of strange comportment, monitor position resetting, panels arrangement reorganizing around default screen, etc. But I'm not sure of this.
Comment 27 romuluspb 2016-07-26 21:35:26 UTC
I also set KillYUserProcesses=yes and removed comments to see if it really avoid the waiting, but it did not worked in my case. I'm using Neon in case.
Comment 28 Harald Sitter 2016-08-09 08:25:33 UTC
I think this should be solved in dev editions within the hour and for user edition some time this week.

http://packaging.neon.kde.org/cgit/neon/settings.git/commit/?id=3a723d1daf55ce4b69b5af56c0a2ef35a31430b6
Comment 29 Harald Sitter 2016-08-09 08:45:40 UTC
nevermind, still broken. won't be solved until plasma learns to clean up and our software learns to not crash in destructors.
Comment 30 Pranav Sharma 2016-08-16 22:19:16 UTC
It seems that the bug is resolved in a newer version of systemd.

https://github.com/systemd/systemd/issues/2691
Comment 31 Harald Sitter 2016-09-21 11:01:09 UTC
The developer editions are about to get a neon-settings update which may prevent this from happening (for as long as it does right now).

It would be awesome if people could test and see if it improves the situation (may need two reboots to apply).
Comment 32 Harald Sitter 2016-09-21 11:12:34 UTC
sudo apt update && sudo apt install neon-settings

then reboot at least twice
Comment 33 Sebastian Kügler 2016-09-21 11:24:08 UTC
Tested and confirmed working, thanks for the fix!
Comment 34 Vishal Rao 2016-09-21 12:17:29 UTC
Appears to be working for me too now. Thank you!

Note on some reboots it still shows "stop job running" but completes at about 7 seconds duration when I set the /etc/systemd/system.conf interval settings to 30 seconds.

And on some reboots it's so fast I don't even get to press ESC during the plymouth shutdown animation to see the console output :-)

Could anyone post a link to the commit or change that works around or fixes this issue, just for the curious?
Comment 35 Harald Sitter 2016-09-28 10:40:49 UTC
I'll file a task to follow up on resolving the underlying causes of this.