diff --git a/skills/wp-s3-media-offload/SKILL.md b/skills/wp-s3-media-offload/SKILL.md new file mode 100644 index 0000000..e253ced --- /dev/null +++ b/skills/wp-s3-media-offload/SKILL.md @@ -0,0 +1,172 @@ +--- +name: wp-s3-media-offload +description: Use when a WordPress site's local wp-content/uploads is growing large or slowing down its host, and needs offloading to the fleet's warrior/stray MinIO pair via WP Offload Media. Covers the plugin (WPS3Media/"WP Offload Media" 4.0.0-cloudhost), the AS3CF_SETTINGS wp-config constant, per-site bucket creation, and the warrior→stray mirror. Also documents an OPEN, unresolved production bug (see bottom) that currently blocks actual uploads through the public endpoint. +--- + +# WP S3 Media Offload (warrior/stray MinIO) + +Pattern used to move a WordPress site's media library off local disk and +onto the fleet's MinIO pair, serving images directly from there instead of +the site's own host. First built for **easycut.es** (cabrera), attempted +for **sateulera-dev** (funky/`sateuleraes` jail) on 2026-08-22 — plugin, +bucket, and mirror are all correctly wired, but **actual uploads are +currently blocked fleet-wide by an open bug**, see the bottom section +before repeating this elsewhere. + +## Architecture + +- **warrior** (`100.78.33.122`, MinIO primary, self-signed cert → `mc` + needs `--insecure`) holds one **dedicated bucket per site** — never + reuse another site's bucket. Bucket policy `download` (anonymous + public-read) so `serve-from-s3` can point directly at objects with no + CDN/signed-URL layer. +- **stray** (`100.118.140.10`, disaster-recovery replica) mirrors that + bucket continuously via the existing systemd template unit + `minio-mirror@.service` (`/etc/systemd/system/minio-mirror@.service`, + already installed on stray — just `mc --insecure mb stray/` then + `systemctl enable --now minio-mirror@.service`). Pull-based + (`mc mirror --watch --insecure warrior/%i stray/%i`), no manual + triggering needed once enabled. +- Both buckets share the **same MinIO access key** fleet-wide (`storgy` — + effectively unrestricted, not bucket-scoped IAM). No new credentials + needed per site. + +## The plugin + +Not the free "as3cf" plugin from wordpress.org — this fleet runs a +**licensed Pro build under the folder name `WPS3Media`** (plugin header +still says "WP Offload Media", version suffixed `-cloudhost`), ~34MB. +There's no cached copy in any git repo — the source of truth is whichever +site already has it installed (currently easycut.es on cabrera, +`/www/wwwroot/easycut.es/wp-content/plugins/WPS3Media`). To add a new +site, `scp -r` that directory straight into the new site's +`wp-content/plugins/`, `chown -R www:www` it, then +`wp plugin activate WPS3Media` (the folder name IS the plugin slug wp-cli +needs here, despite the plugin header reading "WP Offload Media"). + +## wp-config.php wiring + +Add (adjust site name and bucket only — access/secret key are fleet-wide): + +```php +define( 'AS3CF_SETTINGS', serialize( array( + 'provider' => 's3compatible', + 'access-key-id' => '', + 'secret-access-key' => '', +) ) ); +``` + +The rest of the config (bucket name, endpoint, region, prefix) lives in +`wp_options` under the key **`tantan_wordpress_s3`** (no table-prefix +literal in the option_name itself — `wp option` commands add/strip the +site's prefix automatically), not wp-config. Set it via the plugin's own +Settings UI after activation, or headlessly via wp-cli's `--format=json` +(NOT `--format=php` — wp-cli only supports `plaintext`/`json` for `option +update`, passing a hand-serialized PHP string directly gets silently +truncated by shell quoting across an `ssh`→`jexec`→`wp` hop): write a +plain JSON file, `scp` it onto the jail's own filesystem path (not the +host's `/tmp` — a redirect like `< /tmp/x.json` against a `jexec` command +resolves on the HOST before entering the jail), then +`wp option update tantan_wordpress_s3 --format=json < /path/inside/jail/x.json`. +Known-good values (endpoint/region are shared across all sites on this +MinIO instance, only `bucket` changes per site): + +| Key | Value | +|---|---| +| `bucket` | `` | +| `endpoint` | `https://s3.palmasolutions.net` | +| `region` | `eu-south-1` (dummy — required by the plugin's AWS-SDK compat layer, MinIO ignores it) | +| `object-prefix` | `wp-content/uploads/` | +| `serve-from-s3` | `true` | +| `copy-to-s3` | `true` | +| `remove-local-file` | `true` (only once you've confirmed images serve correctly from the bucket) | +| `use-bucket-acls` | `true` | +| `use-yearmonth-folders` | `true` | +| `enable-object-prefix` | `true` | + +## Rollout steps (repeatable) + +1. `mc --insecure mb warrior/` + `mc --insecure anonymous set download warrior/` +2. On stray: `mc --insecure mb stray/` + `systemctl enable --now minio-mirror@.service` +3. **Before wiring the plugin**, if the site is WordPress and has a + `wp-content/uploads/` folder worth shrinking first, run the fleet's + existing webp-sibling pass (`scripts/webp-convert.sh` + deploy + `wp-plugins/granja-webp-siblings.php` as an mu-plugin) — this is + already standard per `docs/new-site-checklist.md` and typically cuts + ~40-54% off what actually needs uploading/storing. +4. Copy `WPS3Media` plugin folder to the target site's `wp-content/plugins/` +5. `wp plugin activate WPS3Media` +6. Set `AS3CF_SETTINGS` in wp-config.php (access/secret key only) +7. Configure bucket/endpoint/region via plugin UI or `wp option update + tantan_wordpress_s3 --format=json < file.json` (see wiring section above) +8. **Try a real signed write BEFORE running the bulk offload against the + whole library** — `mc --insecure cp somefile.txt //test.txt` + using the exact endpoint/credentials the plugin will use. As of + 2026-08-22 this fails with `SignatureDoesNotMatch` for ANY bucket/site + through the public `https://s3.palmasolutions.net` endpoint — see the + "OPEN BUG" section below before assuming a new site's setup is broken; + it almost certainly isn't, the shared endpoint is. +9. Once the endpoint bug above is actually fixed, trigger the offload: + there's no reliable Pro-tools/Uploader-tool wp-cli command in this + "-cloudhost" build (the Tools/license layer returns empty), so drive it + directly instead — loop `get_posts(post_type=attachment)` and call + `wp_update_attachment_metadata($id, wp_get_attachment_metadata($id))` + per ID via `wp eval-file` (this re-fires the exact same hook a real + upload triggers, safely, for existing media). +10. Verify: `mc --insecure ls warrior//wp-content/uploads/` shows + growing object count; front-end pages load images from + `https://s3.palmasolutions.net//...`. Don't flip + `remove-local-file` to true until spot-checked. + +## OPEN BUG (found 2026-08-22, NOT fixed): public endpoint breaks SigV4 + +Every authenticated S3 API request (PUT, even a bodyless LIST) through +`https://s3.palmasolutions.net` (Caddy on madri → warrior) fails with +`SignatureDoesNotMatch`, for ANY valid MinIO credential pair — confirmed +this is not a plugin bug, not a credentials bug, and not a jail-network +bug: + +- Same exact credentials succeed instantly hitting warrior directly + (`https://100.78.33.122:9000` or `https://62.210.214.171:9000`, + bypassing Caddy entirely). +- `storage.palmasolutions.net` (Caddy on the same madri host → stray's + MinIO, over **plain HTTP** to the backend) signs correctly — so this + isn't a generic "Caddy can't front MinIO" problem. +- Two plausible fixes were tried on `infra/madri/Caddyfile`'s + `s3.palmasolutions.net` block and **both disproven by direct retest**: + removing `encode gzip` (no effect — kept removed anyway, no downside) + and forcing `transport http { versions 1.1 }` to rule out Caddy + negotiating HTTP/2 to the HTTPS backend via ALPN (no effect either — + kept anyway as reasonable hardening, but it is NOT the fix). +- `mc admin admin trace -v warrior` failed to connect during this + investigation ("unexpected end of JSON input") — packet-level capture + (tcpdump/mitmproxy comparing the raw bytes MinIO receives via each + path) is the logical next diagnostic step, not more guessing at Caddy + directives. + +**Practical impact**: no WordPress/AS3CF-style plugin (or any other +SigV4 client) can currently write to ANY bucket through +`s3.palmasolutions.net` — only direct-to-warrior access works. This +almost certainly means **easycut.es's own live AS3CF uploads are +currently failing the same way** (its wp-config point at this exact +endpoint) — worth a dedicated check, since that site has a prior security +incident history and any silent failure there deserves attention, not +just this new site's setup. + +Until this is root-caused, treat this skill's "Rollout steps" as +correct-but-blocked at step 8 for any NEW site — the config is right, +the endpoint isn't currently usable. + +## Gotchas + +- Finding the DB socket on panel-managed Linux hosts (cabrera-style + BTPanel/aaPanel installs) is not obvious — `wp-cli` may report "missing + MySQL extension" and the default `/run/mysqld/mysqld.sock` path is + usually wrong. Check `/tmp/mysql.sock` first before assuming MySQL is + down. +- Bastille jails (funky, gringo, phreak) don't have this problem — wp-cli + works normally inside `jexec wp --allow-root --path= ...`. +- Confirm the site's own images aren't already coming from an unrelated + external CDN before assuming local uploads are the bottleneck (sateulera-dev's + *product* images are CloudFront-hosted already — this offload only + applies to the ~950 theme/gallery/local files actually on-disk).