Files
joulenap/backend
Catubba 5b39f9ae1a fix(tests): remove the three races that made the backend suite flaky
The suite failed roughly one full run in five, differently each time. Three
independent causes, not one:

1. The lifespan started `missed-backup-check` and `interrupted-run-alert` as
   daemon threads that nothing joined, and the missed-run check opens a
   database session. Every test that builds an app leaked one, so a thread
   routinely outlived its test and kept using the database while the next test
   tore the engine down and built its own. They are now joined on shutdown,
   which is also the right behaviour in production: a restart no longer
   abandons an alert halfway out. The join is bounded well under Docker's stop
   grace, so a black-holing notification channel still cannot hold the process
   open. The job service's queue worker stays unjoined on purpose - it may be
   mid-backup, and blocking shutdown on a running vzdump is worse.

2. `_ensure_ready()` initialised the database lazily when the session factory
   was missing, which built a schema at `paths.db_path()` - the real database -
   from whichever caller got there first. `create_all` reflects once and then
   issues CREATEs, so two of those at the same time collide with "table X
   already exists", the error the suite reported. It now raises instead, so a
   caller that outlived its setup fails loudly rather than racing. One test was
   relying on that lazy path and gained the fixture it always needed; it was
   the only one, i.e. the only test that had been writing to the real database.

3. The 409 came from a test helper, not from the database race. `_run_route`
   waited for the run row to leave RUNNING, but the row is finalised inside the
   cycle while the power-off, the notification, the single-run lock and the
   queue's current entry all still follow. A second run of the same route
   inside that window is correctly rejected as already queued. The helper now
   waits for the queue itself to go idle.

Both races were reproduced before being fixed - the schema collision by racing
threads through a probe with no initialisation, the 409 by widening the window
with a temporary sleep, which made it fail every run - and both stop
reproducing with the fixes in place.

Two regression guards, each confirmed to fail on the pre-fix code: an autouse
fixture that diffs the live threads around every test and fails the one that
leaves any behind, and a test pinning that using the database before
initialising it raises.

636 passed, 2 skipped; 10 consecutive full runs green, with the real database
untouched.
2026-08-04 14:49:15 +02:00
..