The backup that had been failing for five months
Discovered on 28 August, during a restore I did not need to do. The nightly snapshot had been running since late March and backing up an empty directory.
Read more →Replacing a drive before it failed
smartctl -H said PASSED right up to the day I pulled it, which is the most useful thing I can tell you about smartctl -H.
Read more →DNS is not the problem. Except this time.
Symptom: about one request in three to an internal service timed out. The other two were instant. No pattern by client, by time, or by service.
Read more →The under-stairs thermal problem
Drive temperatures went from a steady 38°C to touching 54°C over four days in late June, which is the sort of number that gets your attention.
Read more →Everything here is a file
Mild irony worth admitting: the log of a self-hosted infrastructure project is a static site with no infrastructure at all.
Read more →Day one
The machine is unremarkable on purpose: a low-power board, 32GB of ECC, four 8TB drives in two mirrored pairs, and a boot SSD that is not part of the array and holds nothing I would miss.
Read more →