service-bundle 5 file formats nosh service-bundle service bundle directories The service bundle mechanism is built from a fundamental mechanism that is upwardly compatible with daemontools and Bruce Guenter's daemontools-encore. service-manager1 manages services, which are defined by two directories somewhere in the filesystem, a supervise directory and a service directory, which do not in themselves require a connection to each other. The service manager itself neither knows nor cares where in the filesystem these directories are. That is the province of the utilities that feed control requests to it. It is not necessary for the relationship between service directories and supervise directories to be one-to-one. One service directory can be shared amongst multiple services, as long as they each have an individual supervise directory. The service-control1 and service-status1 commands directly operate upon services, and share a convention with service-dt-scanner1 that the service directory is the parent directory of the supervise directory. However, it is not necessary for supervise directories to be subdirectories of service directories. Within a service bundle, supervise and service directories are siblings under the bundle's root directory, and these tools also understand that structure. The system-control1 command takes those two subdirectories and places them together into a subtree, forming a service bundle. The contents of the bundle define dependencies and relationships amongst services, controlling both what services system-control1 takes action upon when it is instructed to do a job and how it tells the service manager to plumb the standard inputs and outputs of services together. It is not necessary for the I/O plumbing relationship between services to be exactly one "main" service feeding its output into one subordinate "log" service. The service manager permits arbitrary-length pipelines of services, as well as fan-in. (However, fan-in should be used sparingly as it generally causes more administrative headaches than it solves.) system-control1 makes a distinction between services that are "services" and services that are "targets". Although there is no constraint enforced, and system-control simply follows symbolic links, by convention service bundle directories generally live under /etc/service-bundles/services, /run/service-bundles/services, or /var/service-bundles/services; and target bundle directories generally live under /run/service-bundles/targets or /etc/service-bundles/targets. There is a potential for "services" and "targets" to have the same names. system-control1 allows the bundle name of a service to be suffixed with .service or .target if it is necessary to distinguish between two such conflicting service bundles. As far as basic service management is concerned, there is no difference between "services" and "targets". A service generally has a fully-fledged dæmon process that runs, whereas a target often runs just a dummy placeholder program. A target generally has active dependencies, whereas a service generally only has passive dependencies. There are several conventional "standard targets" used in system administration. A service directory is the current directory in which a service process is run. It contains: a run file, which is the executable file for the service process itself; a start file, which is the executable file to be run when a service is first brought up (but not when it is automatically restarted); a restart file, which is the executable file to be run when a service has ended (to determine whether it should automatically be restarted); a stop file, which is the executable file to be run when a service is finally taken down (but not when it is automatically restarted); Although there is nothing to stop them from being binaries, the executable files are usually scripts interpreted by nosh1, execlineb1, or a shell. They set up various parts of the process state (using commands such as softlimit1, setenv1, setuidgid1, and open-controlling-terminal1) and then chain to the service program proper. A service directory can also contain: ancillary files required by the service itself, varying from service to service. For examples: A tcp-socket-accept1 service could have an access-control database managed by ucspi-socket-rules-check1. Many services have env subdirectories read by envdir1 in order to control dæmon process environment variables. further files used by other tools. For examples: A down file indicates to system-control1, service-is-enabled1, and service-dt-scanner1 that a service should not be auto-started at bootstrap. A remain file indicates to system-control1 that a service should be marked as "run on empty", so that it is considered still running even if it has no processes. A ready_after_run file indicates to system-control1 that a service should be considered "ready" after it has finished running, and has either remained in the running state with no processes or transitioned to the stopped state with a prior run recorded in its status. A use_hangup file indicates to system-control1 that a service should (additionally) be sent the SIGHUP signal when shutting it down. A no_kill_signal file indicates to system-control1 that a service should not be sent the SIGKILL signal when shutting it down. These files are ignored by service-manager. The service manager does not need write access to the service directory or to any of the executables within it. This permits service directories (as long as the services themselves do not require write access to their service directories) to reside on read-only volumes. Automatic restart is tailorable to individual services. If the restart program does not exist, or does not exit with a success (i.e. zero) status when run, the service run program is not restarted. For the simplest cases restart can just be a (symbolic) link to /bin/true or /bin/false, to provide always-restart and never-restart services, respectively. (If using the nosh flavours of true1 and false1 do not use links to them. They will see themselves invoked under the unknown (to them) name restart and complain. Instead, write a short nosh1 script.) However, restart is invoked with two pieces of information, which together represent the most recent exit status of the run program, that allow finer control over the restart decision, if desired. The two pieces of information are its three command line arguments. The first is a code, one of exit, term, kill, abort, or crash. This categorizes how run exited. Everything apart from exit denotes being terminated by an uncaught signal. term denotes the "good" termination signals SIGTERM, SIGPIPE, SIGHUP, and SIGINT. kill denotes SIGKILL. abort denotes SIGABRT, SIGALRM, or SIGQUIT. And crash is everything else. The second is either (for exit) the decimal exit status of the process or (for everything else) a symbolic designation (falling back to a decimal code) of the specific signal, if the first argument is not specific enough to make a decision. For convenience, the third is (for other than exit) always the decimal code of the specific signal. A supervise directory provides the control/status API for the service supervisor. It contains: an ok FIFO that does nothing more than signify that the service manager has loaded the service; a control FIFO through which commands to control the individual service process (for which see service-control1) are sent; a status file that contains a record of the service process ID, start time, and control state; and a lock file (compatible with setlock1) that prevents the service manager from re-using an active supervise directory. Other tools may use further files in a supervise directory. As with extraneous files in service directories, these files are ignored by service-manager. The service manager requires read-write access to these files, and write access to the supervise directory itself, as it creates the files if they do not exist to start with. However, it does not require write access to the supervise directory once the files have been created. (The supervise1 program in daemontools repeatedly re-creates the status file, in contrast.) Control of services and access to service status is thus subject to ordinary permissions and ACLs on these files. Bernstein's daemontools employs an 18-byte status file. daemontools has no notion of "starting", "failing", or "stopping" states for services, and its status file provides only simple binary "up" or "down" state information. Guenter's daemontools-encore employs a 19-byte status that includes extra state information for the aforementioned states. service-manager employs an 87-byte status that adds exit status and timestamp information for the start, run, restart, and stop programs. The status file contents are: 12-byte TAI64N timestamp of last service status change event. 4-byte current main process ID in host byte order, 0 meaning no process and -1 meaning process #0. 1-byte paused flag, if the dæmon process is in the stopped state. 1-byte pending command flag, which is an ASCII-encoded character: ddown uup oonce Oat most once 1-byte daemontools-encore status (for details see later) which is a binary number: 0stopped 1starting 2started 3running 4stopping 5failed 68 bytes comprising 4 groups of status information for the last start, run, restart, and stop programs to terminate: 1-byte code for termination status, which is a binary number: 0 Not yet terminated, other fields should not be considered valid. 1 Terminated normally with an exit code. 2 Terminated by a signal. 3 Terminated by a signal, and core dumped. 4-byte exit code or signal number, in host byte order. 12-byte TAI64N timestamp. If a service is not known to the service manager, it is in an unloaded state, and none of the information in the status file is valid. Otherwise, service states in that file follow the daemontools-encore paradigm: stopped No service process is executing. starting The service's start program is currently executing. started This state is not used. running The service's run program is currently executing, or no program is currently executing in a run-on-empty service that was running the run program immedately prior. failed The service's restart program is currently executing. stopping The service's stop program is currently executing. A bundle directory contains the command information, dependency information, relationship information, and control/status API for a service. It contains various subdirectories (which can be symbolic links to directories elsewhere): The service/ and supervise/ subdirectories are (respectively) the service and supervise directories for the service, comprising the command information and the control/status API. (service-dt-scanner1 implements a slight extension here, for compatibility.) When bundles are on a read-only filesystem, such as a CD-ROM, supervise must be a symbolic link pointing to a directory on a read-write filesystem, since supervise directories must be writable. One might choose a subdirectory of /run/service-bundles/early-supervise/ for example. system-control attempts to create the target of such a symbolic link before attempting to load the service, to avoid a chicken-and-egg situation with "sysinit" services that are set up this way. The wants/, requires/, conflicts/, wanted-by/, stopped-by/, and required-by/ subdirectories comprise dependency or dependency installation information. Each contains a set of symbolic links to other bundle directories for other services. wants/ contains links to services that the current service "wants". Starting the current service implies starting the linked-to service as well. requires/ contains links to services that should be "required-by" the current service when it is enabled. Enabling the service places it in the targets' "required-by" lists, and disabling the service removes it. conflicts/ contains links to services that "conflict" with the current service. Starting the current service implies stopping the linked-to services. required-by/ contains links to services that "require" the current service. Stopping the current service implies stopping the linked-to services as well. wanted-by/ contains links to (standard) targets that should "want" the current service when it is enabled. Enabling the service places it in the targets' wants/ lists, and disabling the service removes it. stopped-by/ contains links to (standard) targets that should "conflict" with the current service when it is enabled. Enabling the service places it in the targets' "conflicts" lists, and disabling the service removes it. The after/ and before/ subdirectories comprise relationship information. Relationships govern the ordering between the individual actions in a start/stop job. after/ contains links to services that should be started after/stopped before the current service when executing a start/stop job. before/ contains links to services that should be started before/stopped after the current service when executing a start/stop job. Relationships are not the same as dependencies. A service can "want" another service without having any ordering against it with respect to start/stop actions. Indeed, this is a common case for system targets. The normal target, for example, wants/ the server, workstation, multi-user, static-networking , and users targets (and by extension everything that they in their turn wants/), but all of the wanted services are started in parallel with the normal target itself. Similarly, a service can be ordered with respect to another service without having a dependency from it. For example, both "log" and "main" services are "wanted" by the server and workstation targets, explicitly. The individual "main" services, therefore, only need to be ordered after their corresponding "log" services, and do not wants/ them. ("log" services are first-class citizens in this respect.) There are no implicit relationships or dependencies in a bundle. The relationships and dependencies are exactly what is explicitly in the filesystem. system-control will fail if the recorded relationships and dependencies are self-contradictory or impossible (such as a service that conflicts with itself, for example). Several system-control1 subcommands take a list of bundle names. Each name can be the full absolute pathname of a bundle directory or a relative pathname with a directory prefix, in which case it is used as-is, or a bundle directory name without any directory prefix, in which case a set of standard locations is searched for a bundle directory by that name. The search algorithm is as follows: A name that ends in .target is searched for (sans the .target suffix) in the standard locations for target bundles: System-wide target directories: /usr/local/etc/service-bundles/targets/ /etc/service-bundles/targets/ /var/local/service-bundles/targets/ /run/service-bundles/targets/ /var/service-bundles/targets/ Per-user target directories: $XDG_RUNTIME_DIR/service-bundles/targets/ $HOME/.config/service-bundles/targets/ A name that ends in .service or .socket is searched for (sans those suffixes) in the standard locations for service bundles: System-wide service directories: /usr/local/etc/service-bundles/services/ /etc/service-bundles/services/ /var/local/service-bundles/services/ /var/local/sv/ (‡) /run/service-bundles/services/ /var/service-bundles/services/ /var/sv/ (‡) /var/svc.d/ (‡) /service/ (‡) Items marked ‡ are searched for backwards compatibility with earlier versions of the nosh toolset, with Wayne Marshall's service directory convention, or with Daniel J. Bernstein's service directory convention. Do not place modern service bundles in these locations unless coexisting with those systems. Per-user service directories: $XDG_RUNTIME_DIR/service-bundles/services/ $HOME/.config/service-bundles/services/ Any other name is searched for (as-is) in the standard locations for service and for target bundles. If the path search fails, the name is simply assumed to name a bundle directory in the current directory. Several standard targets are defined. They conventionally live in the /etc/service-bundles/targets/ directory. Various subcommands of system-control1 employ them for system management. The subcommands send a signal to the system-manager1, which in turn spawns another system-control1 directly commanding the start or stop of one of these standard targets. These standard targets are (indirectly) started by the init subcommand. normal This target is the normal target started by the init subcommand. This should bring up with it all of the services involved in the normal operation of the system, and an administrator-defined set of multiple login user interfaces that may include both TUI and GUI login prompts. It wants/ the workstation, server, and multi-user targets. Thus starting it starts the union of all services wanted by those targets. It is not ordered with respect to those targets. So in the absence of orderings in between the individual services wanted by those targets, defined in their bundles, all wanted workstation, server, and multi-user services are brought up in parallel. emergency This target is the target started by the init subcommand with the -b option. Starting it is intended to bring up the system in "emergency mode", bringing up no services, mounting no (non-system) volumes, and bringing up just "emergency mode" services, such as a secure login on the default terminal. It wants/ no other targets or services. So in "emergency mode" (if nothing else has been started) only the system filesystems are available, and not necessarily even read-write. rescue This target is the target started by the init subcommand with the -s option. Starting it is intended to bring up the system in "rescue mode", bringing up with it the fundamental system initialization services and a secure login user interface. It wants/ the sysinit target, but not the basic target. update This target is the target started by the init subcommand with the --update option. Starting it is intended to bring up the system in "update mode", bringing up with it the fundamental system initialization services and the system update utility. It wants/ the sysinit target, but not the basic target. The poweroff, powercycle, halt, and reboot subcommands invoke several standard targets indirectly. poweroff powercycle halt reboot These four targets both wants/ the shutdown target and order themselves to start after it. Thus starting them shuts down all services and then, once this has been done, starts these targets. They are usually started by the System Manager, in response to system control commands requesting system reboot, halt, or power off. When they (finally) start, they invoke the reboot, halt, powercycle, and poweroff subcommands of system-control with the -f command-line option. This signals the System Manager to enact the actual reboot, halt, power-cycle, or power-off action. Do not start these targets directly with the start, reset, or activate. They will end up terminating the very system-control process that is directing the state change, mid-change. Always use the poweroff, powercycle, halt, and reboot subcommands. For example: When system-manager8 is process #1, system halt operates as follows: The superuser runs system-control halt. This sends SIGUSR1 (on BSD) or SIGRTMIN + 3 (on Linux) to the system manager process. In response to that signal, the system manager runs system-control activate halt. system-control does everything necessary for activating the halt target, including stopping all other (conflicting) services and targets. In the service control ordering starting the halt target comes after processing every other service, because of the ordering between halt and shutdown and between shutdown and everything else. When the halt target is finally started it runs system-control halt -f. This sends SIGRTMIN + 13 to the system manager process. In response to that signal, the system manager performs the finalization action of actually telling the operating system kernel to halt the system, after various standard system shutdown actions such as flushing the filesystem cache. Other standard targets are started by telinit1. User-supplied services and targets that are auto-started at system bootstrap should list themselves in the wants/ and conflicts/ lists of subordinate targets (via the enable and disable subcommands). These targets are usually not started explicitly. workstation All services that normally form part of bringing the system up as a "workstation" should be listed in this target's wants/ list. Such services will include things such as the cron1 dæmon. This target wants/ the basic target, so starting it (directly or indirectly) will bring the basic system up. It is a collection that is used to group services, but is not an ordering milestone and has no before/ or after/ relationships. server All services that normally form part of bringing the system up as a "server" should be listed in this target's wants/ list. Such services will include things such as HTTP/FTP/SMTP dæmons, and dæmons that provide services over the network. This target wants/ the basic target, so starting it (directly or indirectly) will bring the basic system up. It is a collection that is used to group services, but is not an ordering milestone and has no before/ or after/ relationships. multi-user All services that normally form part of bringing up multi-user logins should be listed in this target's wants/ list. Such services will include things such as the login services for terminals (real and virtual) and X. This target wants/ the basic target, so starting it (directly or indirectly) will bring the basic system up. It is a collection that is used to group services, but is not an ordering milestone and has no before/ or after/ relationships. shutdown Starting this target is intended to shut down all running normal services. All normal services should be in this target's conflicts/ list. It is also an ordering milestone, in that all services listed in its conflicts/ list should also be listed in its after/ list. shutdown is then listed in the after/ list of the poweroff, powercycle, halt, and reboot target to ensure that they are only started after the system has entered "shutdown mode". unmount Starting this target is intended to unmount all filesystems, both local and remote. All mount@* services should be in this target's conflicts/ list. unmount is then listed in the after/ list of the poweroff, powercycle, halt, and reboot target to ensure that they are only started after the system has unmounted all filesystems. Core services and targets are incorporated into a set of further subordinate targets. These core targets are solely collections. They are a mechanism for listing a wants/ dependency on a group of services without explicitly enumerating every single member of the group. They are not ordering milestones and have no before/ or after/ relationships. basic This target implies the "basic" system, excluding all workstation, server, and multi-user services. It wants/ the sysinit target. static-networking This target implies the static setup/teardown of network interfaces and routing. It wants/ all of the generated static_protocol@id services. sysinit All fundamental system initialization services and steps should be listed in this target's wants/ list. Such services will include things such as the hostname service that runs set-dynamic-hostname1, the machine-id service that runs setup-machine-id1, the local-fs target, and the remote-fs target. swapauto This target is wanted by "sysinit". All "auto" "swap" services should be listed in its wants/ lists. swaplate This target is wanted by "sysinit". All "late" "swap" services should be listed in its wants/ lists. dumpauto This target is wanted by "sysinit". All "dump" services should be listed in its wants/ lists. finish-install finish-update These targets wants/ the various services that finish a system install or a system upgrade. These core targets are not collections; their wants/ lists should be empty. They are solely ordering milestones, listed in the before/ and after/ lists of other services. They exist because services that need to occur before them do not have another way to hold back the services that need to occur after they become ready. Conventionally, they are brought in only if the services that want to order before/ them are themselves brought in, and it is those services that list these milestones in their wants/ lists. This is only a convention, however. There is no technical reason that it could not be all of the services that want to order after/ the milestones that list them in their wants/ lists. It is simply the case that something must bring the target in, for the orderings to actually take effect. local-fs-pre This "local filesystems precursor" target exists because system initialization services that need to occur before any filesystem checks or mounts, usually special services that initialize an interactive UI for fsck1, do not have another way to hold the checks/mounts from commencing until they are done. Such services should list this target in their wants/ and before/ lists. Local filesystem checks, decryptions, and mounts should list this target in their after/ lists, so that they order relative to it if the target is brought in but do not themselves bring it in. fs-servers This "filesystems servers" target exists in order to ensure that remote filesystem mounts of volumes on the local machine work, by ordering them after the startup of any filesystem servers (e.g. SAMBA or NFS server dæmons). Such server services should list this target in their wants/ and before/ lists. Remote filesystem mounts should list this target in their after/ lists, so that they order relative to it if the target is brought in but do not themselves bring it in. multi-user-pre This "multi-user precursor" target exists for services that need to start/execute before any multi-user login. Such services should list this target in their wants/ and before/ lists. Login services should list this target in their after/ lists. name-services This target exists because some name/address services lookup clients cannot be caused via the client-server communications mechanism itself to block indefinitely until the server is ready; and so have to have an explicit ordering to order them after any respective servers. One lists name-services in the wants/ and after/ list of a service or target to ensure that it is only brought up after such name/address lookup services are available. All services that are involved in providing local name/address lookup services should be listed in its after/. databases This "database servers" target exists because some database clients cannot be caused via the client-server communications mechanism itself to block indefinitely until the server is ready; and so have to have an explicit ordering to order them after any respective servers. One lists databases in the wants/ and after/ lists of a service or target to ensure that it is only brought up after such database services are available. All services that are involved in providing database services should list it in their before/ lists. machine-id One lists machine-id in the wants/ and after/ lists of a service or target to ensure that it is only brought up after the machine ID is available. All services that are involved in setting up the machine ID should list it in their before/ lists. hostname One lists hostname in the wants/ and after/ lists of a service or target to ensure that it is only brought up after the dynamic hostname has been initialized. All services that are involved in initializing up the dynamic hostname should list it in their before/ lists. persistent-clock-loaded This target exists so that time synchronization services can wait until the system clock has been initialized from some internal persistent clock that keeps, or at least statically saves the latest runtime value of, the current time over a powerdown or a restart. All services that are involved in loading the time from internal persistent clocks should list it in their wants/ and before/ lists, and only alter the system clock before they enter the running/ready state, so that they cease controlling the system clock before time synchronization services take over. All services that are involved in time synchronization should list it in just their after/ lists, so that they do not begin controlling the system clock until the hardware clock has been read, and so that they order relative to the target if it is brought in but do not themselves bring it in. system-time-synchronized One lists system-time-synchronized in the wants/ and after/ lists of a service or target to ensure that it is only brought up after the point at which time synchronization services using dynamic external clocks are running. All services that are involved in such time synchronization should list it in their before/ lists. Note that because of the dependencies of the synchronization services themselves, this can potentially involve an ordering after an arbitrary number of other milestones, including the local file system mounts, networking initialization, and virtual machine guest services. Generally, with some exceptions, all services listed in these targets' wants/ lists should also be listed in their after/ lists. local-fs This "local filesystems" target lists all system initialization services and steps for checking and mounting local filesystems, including remounting the root filesystem in read-write mode. One can thus list local-fs in the after/ list of a service or target to ensure that it is only brought up after local filesystems have been mounted and checked. Services encompassed by this target may not rely upon filesystems being checked and mounted, nor upon the root filesystem being mounted read-write. To handle the latter, in particular, their supervise/ subdirectories should be symbolic links to a suitable subdirectory of /run/service-bundles/early-supervise. remote-fs This "remote filesystems" target lists all system initialization services and steps for checking and mounting remote filesystems. One can thus list remote-fs in the after/ list of a service or target to ensure that it is only brought up after remote filesystems have been mounted and checked. reboot-after-install reboot-after-update These target wants/ the reboot target. They are ordering milestones, with all finish-install/finish-update services expected to order themselves before/ these targets. Jonathan de Boyne Pollard