Linux Process Management & systemd

Overview

Process control on Linux happens through signals, and the two that matter most are opposites in one specific way: SIGTERM is a request a process can catch and handle for graceful shutdown, while SIGKILL and SIGSTOP cannot be caught, blocked, or ignored by any process, ever, which is exactly why SIGKILL is the tool of last resort, not the default one. systemd builds service lifecycle management on top of process behavior via its Type= setting, simple considers a unit started the instant the process forks, forking waits for the traditional daemon parent-exits pattern, and notify waits for the service to explicitly report its own readiness, three genuinely different answers to "when is this actually up." ps layers two historical option conventions, UNIX-style and BSD-style, onto the same command, and they change both the selected processes and the displayed columns. The same SIGTERM/SIGKILL mechanics inside a container add a PID 1 complication of their own, see Why Every Container in Your Rolling Deploy Takes an Extra 10 Seconds to Stop; for a broader process/monitoring walkthrough, see Linux Processes and Networking: Signals, Ports, Monitoring.

Quick Reference

Syntax

kill -TERM 4821      # ask the process to shut down gracefully
kill -KILL 4821      # force-terminate, cannot be caught or ignored
systemctl restart myapp.service

Examples

# myapp.service - on-failure restart is the recommended default
# for a long-running service, not "always" or "no".
[Service]
Type=notify
ExecStart=/usr/bin/myapp
Restart=on-failure

systemd doesn't wait forever for a graceful shutdown: systemctl stop sends SIGTERM, then escalates to SIGKILL on its own if the service hasn't exited within TimeoutStopSec (90 seconds by default, system-wide via DefaultTimeoutStopSec=). A service doing meaningful cleanup work needs that budget accounted for explicitly, not assumed to be unlimited:

[Service]
Type=notify
ExecStart=/usr/bin/myapp
TimeoutStopSec=30   # fail fast instead of hanging for the full 90s default

Visual Diagram

Common Mistakes

• Reaching for kill -9 (SIGKILL) as a first move instead of SIGTERM, skipping cleanup logic that mattered.
• Using Type=simple for a service whose own startup takes real time, causing dependent units to start before it's actually ready to handle requests.
• Setting Restart=always on a service that's crash-looping due to a real bug, masking the failure instead of surfacing it.
• Mixing BSD-style and UNIX-style ps options without understanding that each convention changes both which processes are selected and which columns are shown.
• Assuming systemctl stop waits indefinitely for graceful shutdown; it doesn't, TimeoutStopSec (90 seconds by default) forces SIGKILL if the service hasn't exited by then, silently truncating any cleanup logic that needed longer.
• Assuming a signal sent to a containerized process reaches the application the same way it would on a bare host; if the application isn't PID 1 or doesn't have its own signal handling, the container runtime's default SIGTERM/SIGKILL behavior toward PID 1 applies instead, not the application's own handler.

Performance

• Signal delivery itself is essentially instantaneous at the kernel level; the real time cost is however long the receiving process's own handler takes to run before it exits.
• Type=notify avoids systemd either understating or overstating a service's readiness, which matters for dependency-ordered startups where a premature "started" signal causes real request failures downstream.

Best Practices

• Always attempt SIGTERM before SIGKILL, and give a service a real, bounded timeout to shut down cleanly before escalating.
• Choose Type=notify for services that support it, over guessing at simple or forking, since it's the only mode based on the service's own explicit signal.
• Set Restart=on-failure as the default for long-running services rather than always, so a persistently crashing service surfaces as a visible failure instead of restarting forever.
• Pick one ps option convention (BSD or UNIX-style) and use it consistently, rather than mixing them and reasoning about output columns case by case.
• Set TimeoutStopSec deliberately for services with real cleanup work, long enough to finish, short enough that a genuinely stuck service doesn't block a deploy for the full 90-second default.
• In containers, verify the entrypoint process actually forwards signals to the application (or use an init like tini/dumb-init) rather than assuming PID 1 behaves like a normal process's own signal handler.