We had an old version of Ubuntu (12.04) that used sysV. Due to LTS running out for it, we needed to switch to 16.04.
One of the major changes was the change from SysV to systemd. We noticed that when we made this change, all of our code that did any sort of
ssh root@thisismyhostname "shutdown -h now" would hang, when previously it would initiate the shutdown, return EXIT_SUCCESS and close the connection. With systemd however, it seems that the connections remained open. I found the fix to that, which was that we needed to install and enable PAM and had to change from
shutdown -h now to
systemctl poweroff so that systemd would iterate through the connections and close them prior to shutting down, the functionality we want.
The problem that now happens is that probably 1/20th of the time, there seems to be a race condition that happens where the shutdown will get enough CPU to take the system down, which causes SSH to return 255 (the return code for connection closed) instead of returning EXIT_SUCCESS for the shutdown command run from SSH.
We (unfortunately) have more than one place in code that calls shutdown, and I feel like checking for either EXIT_SUCCESS or SSH’s 255 is somewhat of a hacky workaround. Does anyone know of a way to get Ubuntu 16.04 with systemd to return EXIT_SUCCESS after calling
systemctl‘s ability to send its own commands via ssh to remote hosts.
From the man page:
-H, --host= Execute the operation remotely. Specify a hostname, or a username and hostname separated by "@", to connect to. The hostname may optionally be suffixed by a container name, separated by ":", which connects directly to a specific container on the specified host. This will use SSH to talk to the remote machine manager instance. Container names may be enumerated with machinectl -H HOST.
It works a lot like this:
# ssh-copy-id -i ~/.ssh/id_ed25519.pub 172.25.50.100 ... # systemctl -H 172.25.50.100 status ● 172.25.50.100 State: degraded Jobs: 0 queued Failed: 1 units Since: Thu 2017-03-02 01:56:07 EST; 17h ago CGroup: / # systemctl -H 172.25.50.100 poweroff # echo $? 0
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.