Objective
Replace a fragile cron job with a controlled systemd timer using a oneshot service, OnCalendar scheduling, Persistent behavior when appropriate, systemd-analyze validation, journalctl logs, and a simple rollback.
- Do not remove files, packages, containers or logs blindly to regain space or stop a restart loop.
- Capture service status and journal output before restarting repeatedly.
- Record the current state before making a production change and keep a tested rollback path.
- Change one variable at a time. Preserve timestamps, logs and configuration evidence so the result can be explained afterward.
- Use a pilot scope whenever possible and avoid broad service restarts, policy relaxations or destructive cleanup as a first diagnostic step.
Prerequisites
- Root or sudo access and the service, filesystem, container or timer name.
- A backup or configuration copy when the procedure changes persistent service state.
- Administrative access appropriate to the system being changed or diagnosed.
- A clearly identified scope: affected users, systems, addresses, services and the time of the observed problem.
- A maintenance or test window when the procedure can affect production traffic or availability.
- A copy of the current configuration or other recovery material before any irreversible action.
Step-by-step procedure
Establish the baseline and scope
Before changing anything, reproduce the issue or document the requested change on a representative system. Record the affected users or services, exact time, current configuration, recent changes and a known-good comparison point. This baseline is the reference used to decide whether each later step improves the situation.
- The scope and current state are documented well enough to reproduce or verify the procedure.
A controlled systemd timer using a oneshot service
Implement this requirement in a controlled scope: a controlled systemd timer using a oneshot service. Capture the previous value or configuration first, make the smallest change that satisfies the design, and validate the effective state rather than assuming that a saved setting is active. If the expected result is not obtained, stop, restore the previous state, and reassess before expanding the change.
- Evidence for this area is explicit, reproducible and consistent with the intended design.
OnCalendar scheduling
Implement this requirement in a controlled scope: OnCalendar scheduling. Capture the previous value or configuration first, make the smallest change that satisfies the design, and validate the effective state rather than assuming that a saved setting is active. If the expected result is not obtained, stop, restore the previous state, and reassess before expanding the change.
- Evidence for this area is explicit, reproducible and consistent with the intended design.
Persistent behavior when appropriate
Implement this requirement in a controlled scope: Persistent behavior when appropriate. Capture the previous value or configuration first, make the smallest change that satisfies the design, and validate the effective state rather than assuming that a saved setting is active. If the expected result is not obtained, stop, restore the previous state, and reassess before expanding the change.
- Evidence for this area is explicit, reproducible and consistent with the intended design.
Systemd-analyze validation
Implement this requirement in a controlled scope: systemd-analyze validation. Capture the previous value or configuration first, make the smallest change that satisfies the design, and validate the effective state rather than assuming that a saved setting is active. If the expected result is not obtained, stop, restore the previous state, and reassess before expanding the change.
- Evidence for this area is explicit, reproducible and consistent with the intended design.
Journalctl logs
Implement this requirement in a controlled scope: journalctl logs. Capture the previous value or configuration first, make the smallest change that satisfies the design, and validate the effective state rather than assuming that a saved setting is active. If the expected result is not obtained, stop, restore the previous state, and reassess before expanding the change.
- Evidence for this area is explicit, reproducible and consistent with the intended design.
A simple rollback
Implement this requirement in a controlled scope: a simple rollback. Capture the previous value or configuration first, make the smallest change that satisfies the design, and validate the effective state rather than assuming that a saved setting is active. If the expected result is not obtained, stop, restore the previous state, and reassess before expanding the change.
- Evidence for this area is explicit, reproducible and consistent with the intended design.
Validate the complete service
Repeat the original user, system or application workflow from the real source and verify the complete result, not only one command or one local check. Confirm that logs and monitoring show the expected behavior and that no temporary debug, bypass, test account, rule or maintenance setting remains enabled.
- The end-to-end service works or the remaining failure is isolated to a clearly identified component.
Technical commands from the original procedure
These technical blocks are preserved byte-for-byte from the historical procedure and kept in their original order. Review names, addresses, paths and parameters before use.
Technical block 1
systemctl list-timers --allTechnical block 2
crontab -lTechnical block 3
sudo ls -la /etc/cron.d /etc/cron.daily 2>/dev/nullTechnical block 4
sudoedit /etc/systemd/system/boai-job.serviceTechnical block 5
[Unit]
Description=BOAI scheduled job
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
User=<USER>
Group=<GROUP>
WorkingDirectory=<WORKDIR>
ExecStart=/usr/local/sbin/<SCRIPT>Technical block 6
sudo systemctl daemon-reloadTechnical block 7
sudo systemctl start boai-job.serviceTechnical block 8
systemctl status boai-job.service --no-pagerTechnical block 9
journalctl -u boai-job.service -n 50 --no-pagerTechnical block 10
sudoedit /etc/systemd/system/boai-job.timerTechnical block 11
[Unit]
Description=Triggers BOAI job
[Timer]
OnCalendar=Mon.Fri *-*-* 03:15:00
Persistent=true
RandomizedDelaySec=10m
Unit=boai-job.service
[Install]
WantedBy=timers.targetTechnical block 12
systemd-analyze calendar 'Mon.Fri *-*-* 03:15:00'Technical block 13
timedatectlTechnical block 14
sudo systemctl daemon-reloadTechnical block 15
sudo systemctl enable --now boai-job.timerTechnical block 16
systemctl list-timers boai-job.timer --allTechnical block 17
systemctl status boai-job.timer --no-pagerTechnical block 18
sudo systemctl start boai-job.serviceTechnical block 19
journalctl -u boai-job.service --since '-10 min' --no-pagerTechnical block 20
systemctl --failedTechnical block 21
journalctl -u boai-job.service --since todayTechnical block 22
journalctl --disk-usageTechnical block 23
sudo systemctl disable --now boai-job.timerTechnical block 24
sudo systemctl enable --now boai-job.timerValidation
The procedure is validated when:
- The original symptom or change request has been tested end to end.
- The effective configuration matches the intended design and no unexplained error remains in the relevant logs.
- Temporary troubleshooting controls have been removed and monitoring remains normal.
- The result, evidence and any follow-up action are documented.
Rollback
- Restore the configuration, policy, binding, route, credential assignment or service state recorded in the baseline when the change does not meet its success criteria.
- Remove temporary rules, test objects and diagnostic settings that were introduced only for the procedure.
- After rollback, repeat the minimum health checks to confirm that the previous service level has been restored.
Troubleshooting / common errors
- A deleted file can still consume disk space while held open by a process.
- Systemd and container restart policies can hide the first meaningful failure; inspect the earliest relevant log entry.
- If the result changes between tests, compare source, destination, identity, time and policy context before changing additional settings.
- If a command succeeds but the application still fails, continue at the next protocol or application layer instead of widening access.
- If the expected evidence is missing, verify that logging, auditing and the test path actually cover the failing component.
- If the change does not improve the measured symptom, restore the previous state and reassess the working hypothesis.
Official and vendor references preserved from the original procedure
- https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html
- https://www.freedesktop.org/software/systemd/man/latest/systemd.time.html
- https://www.freedesktop.org/software/systemd/man/latest/systemctl.html
Adapt names, addresses, paths and parameters to your environment. Back up the configuration and test on a limited scope first.