Managing Daemons with phd

Phorge Administrator and User Documentation (Configuration)

Explains Phorge daemons and the daemon control program phd.

Overview

Phorge uses daemons (background processing scripts) to handle a number of tasks:

• tracking repositories, discovering new commits, and importing and parsing commits;
• sending email; and
• collecting garbage, like old logs and caches.

Daemons are started and stopped with phd (the Phabricator Daemon launcher). Daemons can be monitored via a web console.

You do not need to run daemons for most parts of Phorge to work, but some features (principally, repository tracking with Diffusion) require them and several features will benefit in performance or stability if you configure daemons.

phd

phd is a command-line script (located at phorge/bin/phd). To get a list of commands, run phd help:

phorge/ $ ./bin/phd help
NAME
        phd - phorge daemon launcher
...

Generally, you will use:

• phd start to launch all daemons;
• phd restart to restart all daemons;
• phd status to get a list of running daemons; and
• phd stop to stop all daemons.

If you want finer-grained control, you can use:

• phd launch to launch individual daemons; and
• phd debug to debug problems with daemons.

Automatically start phd

Computers are good in automatically starting stuff, thanks to the invention of the "init system".

Phorge virtually supports any init system. Which one is yours? Don't worry. If you don't know, it's systemd.

Phorge ships with a systemd configuration file, following some assumptions:

• Your lovely Phorge is installed /somewhere/phorge.
• Your repository storage location is /somewhere/repositories.
• You have a dedicated Unix user called daemon-user - coming from Diffusion User Guide.

First, link the phd service into your system configuration. This will ensure that when you update Phorge, the daemon service configuration is updated as well.

systemctl link /somewhere/phorge/resources/phd/phorge-phd.service

Create the following configuration so the service knows where Phorge is:

PHORGE_ROOT=/somewhere/phorge

Next, configure the service to use your daemon user, and give it access to your repository storage path. The following command will open up your default editor:

systemctl edit phorge-phd.service

In the editor, between the comments indicated in the file, place the following to set the user/group phd shall run as, along with the filesystem path(s) for Phorge's repository storage and the phd log location. The only paths you must include in ReadWritePaths are the paths in your phorge configuration that phd will write to:

• repository.default-local-path
• phd.log-directory (Shown below as /var/log/phorge/phd; use the path you set in your config.)

If have not explicitly set phd.log-directory the logs are placed in a temporary directory that is removed every time phorge-phd.service terminates and you do not need to configure ReadWritePaths for it. Even if you do not use the systemd service, the temporary directory phd uses by default may still be removed upon system reboot. If you care about being able to easily diagnose problems, set the option to ensure phd logs persist.

[Service]
User=daemon-user
Group=daemon-user

ReadWritePaths=/somewhere/repositories
# uncomment or remove depending on phd.log-directory option.
#ReadWritePaths=/var/log/phorge/phd

If your database service is managed by systemd and running on the same machine as Phorge (unlikely except in development environments), you may also wish to configure a dependency on it so that phd starts more reliably.

Dependencies are configured by adding Wants and After systemd directives. Consult man 5 systemd.unit for the particulars of what those mean, but we'll add them with systemctl edit phorge-phd.service once again.

If you're using MariaDB:

Add the following to the top of the configuration you created above:

[Unit]
Wants=mariadb.service
After=mariadb.service

The full file should look like the following when you're done:

[Unit]
Wants=mariadb.service
After=mariadb.service

[Service]
User=daemon-user
Group=daemon-user

ReadWritePaths=/somewhere/repositories
# uncomment or remove depending on phd.log-directory option.
#ReadWritePaths=/var/log/phorge/phd

For other MySQL flavors, consult your system configuration for the appropriate unit name.

To install this new systemd configuration, execute these commands as super-user:

systemctl enable --now phorge-phd

Now the process has started and will survive after any reboot.

To check if everything is OK:

systemctl status phorge-phd

Anything else can be explored in depth by reading the systemd documentation. Applicable systemd man pages are systemd.exec(5) and systemd.service(5), which can be scrutinized for details on how how to expand or restrict phd's system access beyond the reasonably restricted defaults.

Daemon Console

You can view status and debugging information for daemons in the Daemon Console via the web interface. Go to /daemon/ in your install or navigate to Home → More Applications → Daemons.

The Daemon Console shows a list of all the daemons that have ever launched, and allows you to view log information for them. If you have issues with daemons, you may be able to find error information that will help you resolve the problem in the console.

Available Daemons

You can get a list of launchable daemons with phd list:

• test daemons are not generally useful unless you are developing daemon infrastructure or debugging a daemon problem;
• PhabricatorTaskmasterDaemon performs work from a task queue;
• PhabricatorRepositoryPullLocalDaemon daemons track repositories, for more information see Diffusion User Guide; and
• PhabricatorTriggerDaemon schedules event triggers (see Understanding Event Triggers) and cleans up old logs and caches.

Debugging and Tuning

In most cases, phd start handles launching all the daemons you need. However, you may want to use more granular daemon controls to debug daemons, launch custom daemons, or launch special daemons like the IRC bot.

To debug a daemon, use phd debug:

phorge/bin/ $ ./phd debug <daemon>

You can pass arguments like this (normal arguments are passed to the daemon control mechanism, not to the daemon itself):

phorge/bin/ $ ./phd debug <daemon> -- --flavor apple

In debug mode, daemons do not daemonize, and they print additional debugging output to the console. This should make it easier to debug problems. You can terminate the daemon with ^C.

To launch a nonstandard daemon, use phd launch:

phorge/bin/ $ ./phd launch <daemon>

This daemon will daemonize and run normally.

General Tips

• You can set the maximum number of taskmasters that will run at once by adjusting phd.taskmasters. If you have a task backlog, try increasing it.
• When you phd launch or phd debug a daemon, you can type any unique substring of its name, so phd launch pull will work correctly.
• phd stop and phd restart stop all of the daemons on the machine, not just those started with phd start. If you're writing a restart script, have it launch any custom daemons explicitly after phd restart.
• You can write your own daemons and manage them with phd by extending PhabricatorDaemon. See Adding New Classes.
• See Diffusion User Guide for details about tuning the repository daemon.

Multiple Hosts

For information about running daemons on multiple hosts, see Cluster: Daemons.

Next Steps

Continue by:

• learning about the repository daemon with Diffusion User Guide; or
• writing your own daemons with Adding New Classes.