Service Management (systemd)

Skill Level: Intermediate

1. Overview

You have already been introduced to systemd and the fundamentals of service management. Now we’ll progress to the more sophisticated capabilities of systemd and customization of the services with additional functionality.

2. Getting Started

For these exercises, you will be using the host node1 as user root.

From host bastion, ssh to node1.

ssh node1

Use sudo to elevate your privileges.

[[ "$UID" == 0 ]] || sudo -i

Verify that you are on the right host for these exercises.

workshop-systemd-checkhost.sh

You are now ready to proceed with these exercises.

3. Customize Default Services

systemd controls more than just daemons or services. For this lab, we will primarily be working with service units but it’s important to know that systemd is handling the dependencies between other types of units:

  • sockets

  • timers

  • mounts

  • swap

  • slices

  • more…​

Unit files are stored in one of three places:

  • '/usr/lib/systemd/system' - default configs that ship with the RHEL and are updated by regular maintenance

  • '/etc/systemd/system' - custom configs that persist and replace (or augment) default configs

  • '/run/systemd/system' - runtime changes that won’t persist

While the defaults for unit files won’t need to be altered most of the time, there will be circumstances where changing the defaults is quite beneficial. These could include hardware or software watchdog monitoring, tunings, resource management, or many other reasons.

3.1. Modify httpd

Create a drop-in configuration file to extend the default httpd.service unit

workshop-systemd-httpdconfig.sh

Contents of /etc/systemd/system/httpd.service.d/50-httpd.conf

[Service]
Restart=always
OOMScoreAdjust=-1000

OOMScoreAdjust is used by the Kernel’s Out Of Memory killer and is an integer between -1000 (to disable OOM killing for this process) and 1000 (to make killing of this process under memory pressure very likely).

Notify systemd of the changes.

systemctl daemon-reload

3.2. Modify mariadb

Similar to what you did in the last step, extend the mariadb.service unit with Restart=always.

This time we’ll use systemctl to create the drop-in and notify systemd of the changes.

systemctl edit allows inserting the content for the drop-in and also handles the systemctl daemon-reload automatically.

You must add your service customizations BETWEEN the commented lines near the top of the template, otherwise your update will be discarded

Your edit should look like this before you save it.

### Editing /etc/systemd/system/mariadb.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file

[Service]
Restart=always

### Lines below this comment will be discarded

Proceed to make your changes now.

systemctl edit mariadb

Type (cut & paste) the following using the automatically launched editor (vi) between the commented lines as outlined above:

[Service]
Restart=always

Save and quit the editor (type :wq , then press <ENTER>) then view the unit

systemctl cat is a quick and easy way to view the contents of a unit and its drop-ins.

systemctl --no-pager cat mariadb
...snip...
# Restart crashed server only, on-failure would also restart, for example, when
# my.cnf contains unknown option
Restart=on-abort
RestartSec=5s

UMask=007

# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec=300

# Place temp files in a secure directory, not /tmp
PrivateTmp=true

# /etc/systemd/system/mariadb.service.d/override.conf
[Service]
Restart=always

3.3. Verify

systemctl --no-pager status httpd
● httpd.service - The Apache HTTP Server
     Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; preset: disabled)
    Drop-In: /etc/systemd/system/httpd.service.d
             └─50-httpd.conf
     Active: active (running) since Mon 2023-09-25 14:53:46 UTC; 5min ago
       Docs: man:httpd.service(8)
   Main PID: 36188 (httpd)
     Status: "Total requests: 0; Idle/Busy workers 100/0;Requests/sec: 0; Bytes served/sec:   0 B/sec"
      Tasks: 213 (limit: 22480)
     Memory: 33.3M
        CPU: 336ms
     CGroup: /system.slice/httpd.service
             ├─36188 /usr/sbin/httpd -DFOREGROUND
             ├─36233 /usr/sbin/httpd -DFOREGROUND
             ├─36238 /usr/sbin/httpd -DFOREGROUND
             ├─36239 /usr/sbin/httpd -DFOREGROUND
             └─36252 /usr/sbin/httpd -DFOREGROUND

Notice that systemctl status displays that the unit has been extended with a drop-in file.

systemctl --no-pager status mariadb
● mariadb.service - MariaDB 10.5 database server
     Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; preset: disabled)
    Drop-In: /etc/systemd/system/mariadb.service.d
             └─override.conf
     Active: active (running) since Mon 2023-09-25 14:53:48 UTC; 6min ago
       Docs: man:mariadbd(8)
             https://mariadb.com/kb/en/library/systemd/
   Main PID: 36509 (mariadbd)
     Status: "Taking your SQL requests now..."
      Tasks: 8 (limit: 22480)
     Memory: 73.1M
        CPU: 456ms
     CGroup: /system.slice/mariadb.service
             └─36509 /usr/libexec/mariadbd --basedir=/usr

4. Conclusion

Hopefully you should now have a fundamental understanding of how services are installed, managed and customized on Red Hat Enterprise Linux 10.

Time to finish this unit and return the shell to its home position.

workshop-finish-exercise.sh

5. Additional Resources

You can find more information:

End of Unit