How temporary data loss helped me improve my backup policiesPublished on Sun 29 March 2015.
Two recent situations have motivated me to rethink and improve my backup policies. Once I have accidentally clicked the ‘Mark as read’ button in my feed reader which marked all 5 000+ unread entries as read, not only the ones from a single feed where I intended to do this. During another day, I have decided to reboot my phone. Then I spent thirty minutes trying various disk encryption passphrases that might be able to unlock it: I finally succeeded, knowing that I guessed correctly that I will forget it when I rebooted it previously a year and a half ago.
I have solved the unread feed issue in three ways: by removing the ‘Mark as read’ button from my installation of Tiny Tiny RSS, by exporting relevant data from a week-old database in my usual server backup to my file with bookmarks from browsers, and by manually marking as unread newer articles. (Why I’m not just marking them as read to shorten my backlog: there are very useful articles among them, often in areas that I don’t need to learn about today, but that I might need in future.)
The database backup wasn’t necessarily consistent and working, but I
was able to use it for this purpose. Since that day, I have
configured daily backup of all my PostgreSQL databases to a git repo
pg_dumpall. Unlike possibly inconsistent files being written
to while the filesystem backup is made, SQL dumps made by
are consistent while other processes write to the databases at the
same time. (This isn’t a proper use of git: with a 230+ MiB file
changed daily, the repo grows quickly and
git gc by default won’t
work since it will run out of memory. Two
git repack options fixed
pack.windowMemory=100m, resulting in
76 MiB repository for backup from 18 days.)
Disk encryption protects only systems powered off. It’s obviously worthless on my phone, since it wasn’t powered off for a year and half (longer than these are expected to work outside of the EU). At the same time, it vastly increases my risk of data loss: I remember passwords by typing them, forget by not typing. The solution is to not encrypt phone’s storage, turn all systems with disk encryption off for each night, so I type their passphrases daily, and keep the passphrases written.
This won’t help if I lose the phone itself or if its eMMC chip
corrupts itself and destroys the files. For this I need regular
backups of its whole
/data partition. I should also schedule
regularly moving various files from it to my other systems (like
photos that I make using the phone and later store on my other
computers, or reading progress for various ebooks).
I have much other data that I haven’t lost yet, even for a short time like 30 minutes of typing passphrases. I should get trivial data that I’m much more likely to independently lose in the same ways as my important files, so I can practice restoring it and make my backup policies reliable. A more certain way is to not delete information, storing all changes of important state in a DVCS-like way (with replication and separate backup), and actively use restored backups.
Submit comments on this post to <firstname.lastname@example.org>.