Cron jobs & systemd timers fail silently #3

New Issue

greg · 2019-01-02T11:45:35Z

greg commented

2019-01-02 11:45:35 +00:00

We don't get any notifications for failed cron jobs, so we can't fix things like e.g. LE certs not updating

Moved from https://gitlab.com/kosmos/chef/issues/21

We don't get any notifications for failed cron jobs, so we can't fix things like e.g. LE certs not updating Moved from https://gitlab.com/kosmos/chef/issues/21

greg changed title from ~~Cron jobs fail silently~~ to Cron jobs & systemd timers fail silently

2019-06-17 09:15:01 +00:00

greg commented

2019-06-17 09:16:02 +00:00

We're using systemd timers to run certbot, so now this issue is about both cron jobs and system timers. I'm going to work on that one

greg referenced this issue from a commit

2019-06-18 16:27:36 +00:00

Replace the certbot systemd unit with a cron job with notifications Uses cronic (https://habilis.net/cronic/) to filter out the stdout and send a sensible email to ops@kosmos.org when renewal fails Closes #3

greg referenced this issue from a commit

2019-06-20 10:48:23 +00:00

Add email notifications for failed certbot runs Based on https://wiki.archlinux.org/index.php/Systemd/Timers#MAILTO This can easily be used by other services, with one line added to the [Unit] section of a service: OnFailure=status-email-ops@%n.service Refs #3

raucao commented

2019-07-02 18:48:22 +00:00

After having implemented the email notifications, this issue needs updating about which cron jobs and timers are left, that can fail silently.

greg commented

2019-08-01 10:09:38 +00:00

I checked on barnard and andromeda, the only cron job is backup (already notifies on failures/warnings) and the certbot timer is the only one that I found that needs email notifications. Closing this one

greg closed this issue

2019-08-01 10:09:39 +00:00

Sign in to join this conversation.

No Milestone

No project

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: kosmos/chef#3