docs: document pf NAT interface-macro silent-no-op bug on shared-IP-alias hosts
Real incident on the staging host: nat on $ext_if from <subnet> to any -> ($ext_if) silently failed to rewrite the source IP when that IP was already aliased on the same interface, breaking all outbound jail traffic with no error anywhere - just clean timeouts. Fixed by using an explicit target IP instead of the interface macro. Cost real debugging time to trace from "WordPress admin hangs after a PHP upgrade" back to this. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -169,6 +169,59 @@ If a host doesn't show this pattern on ANY existing jail, it likely
|
||||
doesn't use per-jail dedicated ports — don't assume the convention
|
||||
applies fleet-wide without checking the specific host first.
|
||||
|
||||
## pf `nat on $ext_if from <subnet> to any -> ($ext_if)` silently fails to translate the source IP when that IP is already aliased on the same interface
|
||||
|
||||
**Confirmed real incident, 2026-08-07**, on a host using the shared-IP-
|
||||
aliasing jail pattern (jail IPs are `/32` aliases directly on the host's
|
||||
own external interface, e.g. `192.168.0.185` aliased onto `vtnet0`
|
||||
alongside the interface's real address `192.168.168.64` -- not the VNET/
|
||||
bridge pattern where jails get a genuinely separate subnet). A `nat on
|
||||
$ext_if from <jail-subnet> to any -> ($ext_if)` rule -- the standard,
|
||||
seemingly-correct pf idiom for masquerading outbound jail traffic --
|
||||
**did not rewrite the source address at all** for these jails. Outbound
|
||||
packets left with the jail's own 192.168.0.x source IP untouched (only
|
||||
the source *port* got NAT'd), which the upstream network silently
|
||||
dropped (source IP outside the VM's assigned subnet, likely anti-spoof
|
||||
filtering) -- symptom was a clean connection timeout, not a pf/rule
|
||||
error, with a real `SYN_SENT` state visible in `pfctl -s states` showing
|
||||
the untranslated source IP in parens. This broke every outbound call any
|
||||
jail on that subnet tried to make (external API calls, WordPress core's
|
||||
own update-check pings, etc.) -- surfaced as WordPress admin pages
|
||||
hanging/timing out with no PHP error logged anywhere, since the PHP-FPM
|
||||
worker was just sleeping on a TCP connect that would never complete.
|
||||
|
||||
**Root cause**: the `($ext_if)` interface-macro NAT target apparently
|
||||
skips/no-ops translation when the connection's source address is *already*
|
||||
one of the interface's own configured addresses -- plausible as "this
|
||||
traffic looks locally-originated already, no need to rewrite" logic in
|
||||
pf's NAT implementation, but wrong for this topology where those aliased
|
||||
addresses are jail IPs that specifically need translating to look like
|
||||
the host's *primary* address before leaving.
|
||||
|
||||
**Fix: use an explicit IP as the NAT target, not the interface macro.**
|
||||
```
|
||||
ext_if = "vtnet0"
|
||||
ext_ip = "192.168.168.64" # the interface's real, non-aliased address
|
||||
lan_net = "192.168.0.0/24" # the jail-alias subnet
|
||||
|
||||
nat on $ext_if from $lan_net to any -> $ext_ip # works
|
||||
# nat on $ext_if from $lan_net to any -> ($ext_if) # looked identical, silently no-ops
|
||||
```
|
||||
**Verify a NAT fix actually rewrites the source**, don't just check
|
||||
`pfctl -s nat` shows the rule (a no-op-translating rule still shows up
|
||||
there looking correct) -- watch `pfctl -s states` during a live outbound
|
||||
attempt from the jail and confirm the parenthetical translated
|
||||
address/port shown is genuinely different from the jail's own IP, not
|
||||
just a port change on the same source IP.
|
||||
|
||||
This is a distinct topology from the VNET/bridge pattern documented
|
||||
elsewhere in this skill and in `jail-dedicated-subnet-migration` (where
|
||||
jails get a real separate subnet on a bridge interface, e.g.
|
||||
`10.20.0.0/24`) -- that pattern's NAT rules were unaffected by this bug
|
||||
in the same test. Check which pattern a given host actually uses
|
||||
(`ifconfig` -- are jail IPs aliased directly on the external interface,
|
||||
or on a separate bridge?) before assuming either NAT idiom is safe.
|
||||
|
||||
## Stock stack includes Valkey, not just nginx/php/MariaDB
|
||||
|
||||
The fleet's standard per-host stack (see `docs/server-funky.md`) is
|
||||
|
||||
Reference in New Issue
Block a user