bastille-jail-provisioning: how to actually change bastille_prefix
Records the gringo 2026-09-06 migration: zfs set mountpoint fails busy with child datasets mounted, zfs set -u avoids it entirely, and jail.conf (not just fstab) carries the path -- missing it means zero jails start.
This commit is contained in:
@@ -280,6 +280,55 @@ writing to an arbitrary path. **Check the host's actual
|
||||
`bastille_prefix`** (`grep bastille_prefix /etc/rc.conf` or
|
||||
`bastille.conf`) before hardcoding either path in a script.
|
||||
|
||||
## Changing bastille_prefix on a host with running jails
|
||||
|
||||
Done successfully on gringo 2026-09-06 (76 jails, 112 GB,
|
||||
`/usr/local/bastille` -> `/www/bastille`). The naive approach fails and the
|
||||
working one is not obvious, so both are recorded.
|
||||
|
||||
**What does NOT work:** `zfs set mountpoint=<new> zroot/bastille` with the
|
||||
jails stopped. It fails with `cannot unmount '/usr/local/bastille': pool or
|
||||
dataset is busy` -- a parent cannot unmount while its child datasets
|
||||
(`jails`, `releases`, `cache`, `backups`, `templates`) are mounted beneath
|
||||
it. Unmounting the tree deepest-first does not reliably clear it either.
|
||||
Attempting this cost a ~7 minute outage for nothing.
|
||||
|
||||
**What works:**
|
||||
|
||||
1. `zfs set -u mountpoint=<new> <dataset>` for each dataset whose mountpoint
|
||||
source is **`local`**. `-u` sets the property *without* unmounting, so it
|
||||
cannot hit the busy error, and the jails keep running throughout.
|
||||
**Only the local ones need setting** -- check with
|
||||
`zfs get -r -H -o name,value,source mountpoint zroot/bastille`. On gringo
|
||||
that was 8 datasets; the other 153 inherit from `zroot/bastille/jails` and
|
||||
follow automatically. Watch for a child deliberately mounted *outside* the
|
||||
tree (`zroot/bastille/logs` at `/var/log/bastille`) -- leave it alone.
|
||||
2. Rewrite the config files. **There are three kinds, and missing one leaves
|
||||
every jail unable to start:**
|
||||
- `bastille.conf` -- `bastille_prefix`
|
||||
- `<prefix>/jails/*/fstab` -- the nullfs release-mount line
|
||||
- **`<prefix>/jails/*/jail.conf` -- `mount.fstab` and `path`.** This is the
|
||||
one that is easy to forget. Rewriting only `fstab` and rebooting gives
|
||||
every jail `jail: <name>: mount.fstab: /old/path/fstab: No such file or
|
||||
directory` and **zero jails start**.
|
||||
- plus any `rc.d` script that hardcodes the prefix.
|
||||
3. Reboot the host. The pool mounts fresh from the properties; nothing has to
|
||||
be force-unmounted. Jails then autostart if `bastille_enable=YES`.
|
||||
|
||||
**Before rebooting a jail host, confirm two things:**
|
||||
- **Which tailscaled owns your session.** `ps -o jid,command -ax | grep tailscaled`
|
||||
-- if the one serving your address has a non-zero JID it is *inside a jail*,
|
||||
and stopping jails cuts your own access. On gringo the host's own (JID 0)
|
||||
owns the address, but a jailed instance existed too.
|
||||
- **A real console path.** Find the VM with
|
||||
`pvesh get /cluster/resources --type vm` from any cluster node and test
|
||||
`qm status <vmid>` before you need it. Server docs go stale -- gringo's said
|
||||
node `sunshine` / 8 GB when it was actually node `informatiq` / 16 GB.
|
||||
|
||||
`bastille start ALL` starts *every* jail, including ones deliberately stopped.
|
||||
Record `jls -N` output **to a durable path, not /tmp** (a reboot clears it)
|
||||
before starting, and re-stop the extras afterwards.
|
||||
|
||||
## Provisioning script safety
|
||||
|
||||
Applies to any shell script written to automate part of this
|
||||
|
||||
Reference in New Issue
Block a user