--- name: wp-s3-media-offload description: Use when a WordPress site's local wp-content/uploads is growing large or slowing down its host, and needs offloading to the fleet's warrior/stray MinIO pair via WP Offload Media. Covers the plugin (WPS3Media/"WP Offload Media" 4.0.0-cloudhost), the AS3CF_SETTINGS wp-config constant, per-site bucket creation, and the warrior→stray mirror. Also documents an OPEN, unresolved production bug (see bottom) that currently blocks actual uploads through the public endpoint. --- # WP S3 Media Offload (warrior/stray MinIO) Pattern used to move a WordPress site's media library off local disk and onto the fleet's MinIO pair, serving images directly from there instead of the site's own host. First built for **easycut.es** (cabrera), attempted for **sateulera-dev** (funky/`sateuleraes` jail) on 2026-08-22 — plugin, bucket, and mirror are all correctly wired, but **actual uploads are currently blocked fleet-wide by an open bug**, see the bottom section before repeating this elsewhere. ## Architecture - **warrior** (`100.78.33.122`, MinIO primary, self-signed cert → `mc` needs `--insecure`) holds one **dedicated bucket per site** — never reuse another site's bucket. Bucket policy `download` (anonymous public-read) so `serve-from-s3` can point directly at objects with no CDN/signed-URL layer. - **stray** (`100.118.140.10`, disaster-recovery replica) mirrors that bucket continuously via the existing systemd template unit `minio-mirror@.service` (`/etc/systemd/system/minio-mirror@.service`, already installed on stray — just `mc --insecure mb stray/` then `systemctl enable --now minio-mirror@.service`). Pull-based (`mc mirror --watch --insecure warrior/%i stray/%i`), no manual triggering needed once enabled. - Both buckets share the **same MinIO access key** fleet-wide (`storgy` — effectively unrestricted, not bucket-scoped IAM). No new credentials needed per site. ## The plugin Not the free "as3cf" plugin from wordpress.org — this fleet runs a **licensed Pro build under the folder name `WPS3Media`** (plugin header still says "WP Offload Media", version suffixed `-cloudhost`), ~34MB. There's no cached copy in any git repo — the source of truth is whichever site already has it installed (currently easycut.es on cabrera, `/www/wwwroot/easycut.es/wp-content/plugins/WPS3Media`). To add a new site, `scp -r` that directory straight into the new site's `wp-content/plugins/`, `chown -R www:www` it, then `wp plugin activate WPS3Media` (the folder name IS the plugin slug wp-cli needs here, despite the plugin header reading "WP Offload Media"). ## wp-config.php wiring Add (adjust site name and bucket only — access/secret key are fleet-wide): ```php define( 'AS3CF_SETTINGS', serialize( array( 'provider' => 's3compatible', 'access-key-id' => '', 'secret-access-key' => '', ) ) ); ``` The rest of the config (bucket name, endpoint, region, prefix) lives in `wp_options` under the key **`tantan_wordpress_s3`** (no table-prefix literal in the option_name itself — `wp option` commands add/strip the site's prefix automatically), not wp-config. Set it via the plugin's own Settings UI after activation, or headlessly via wp-cli's `--format=json` (NOT `--format=php` — wp-cli only supports `plaintext`/`json` for `option update`, passing a hand-serialized PHP string directly gets silently truncated by shell quoting across an `ssh`→`jexec`→`wp` hop): write a plain JSON file, `scp` it onto the jail's own filesystem path (not the host's `/tmp` — a redirect like `< /tmp/x.json` against a `jexec` command resolves on the HOST before entering the jail), then `wp option update tantan_wordpress_s3 --format=json < /path/inside/jail/x.json`. Known-good values (endpoint/region are shared across all sites on this MinIO instance, only `bucket` changes per site): | Key | Value | |---|---| | `bucket` | `` | | `endpoint` | `https://s3.palmasolutions.net` | | `region` | `eu-south-1` (dummy — required by the plugin's AWS-SDK compat layer, MinIO ignores it) | | `object-prefix` | `wp-content/uploads/` | | `serve-from-s3` | `true` | | `copy-to-s3` | `true` | | `remove-local-file` | `true` (only once you've confirmed images serve correctly from the bucket) | | `use-bucket-acls` | `true` | | `use-yearmonth-folders` | `true` | | `enable-object-prefix` | `true` | ## Rollout steps (repeatable) 1. `mc --insecure mb warrior/` + `mc --insecure anonymous set download warrior/` 2. On stray: `mc --insecure mb stray/` + `systemctl enable --now minio-mirror@.service` 3. **Before wiring the plugin**, if the site is WordPress and has a `wp-content/uploads/` folder worth shrinking first, run the fleet's existing webp-sibling pass (`scripts/webp-convert.sh` + deploy `wp-plugins/granja-webp-siblings.php` as an mu-plugin) — this is already standard per `docs/new-site-checklist.md` and typically cuts ~40-54% off what actually needs uploading/storing. 4. Copy `WPS3Media` plugin folder to the target site's `wp-content/plugins/` 5. `wp plugin activate WPS3Media` 6. Set `AS3CF_SETTINGS` in wp-config.php (access/secret key only) 7. Configure bucket/endpoint/region via plugin UI or `wp option update tantan_wordpress_s3 --format=json < file.json` (see wiring section above) 8. **Try a real signed write BEFORE running the bulk offload against the whole library** — `mc --insecure cp somefile.txt //test.txt` using the exact endpoint/credentials the plugin will use. As of 2026-08-22 this fails with `SignatureDoesNotMatch` for ANY bucket/site through the public `https://s3.palmasolutions.net` endpoint — see the "OPEN BUG" section below before assuming a new site's setup is broken; it almost certainly isn't, the shared endpoint is. 9. Once the endpoint bug above is actually fixed, trigger the offload: there's no reliable Pro-tools/Uploader-tool wp-cli command in this "-cloudhost" build (the Tools/license layer returns empty), so drive it directly instead — loop `get_posts(post_type=attachment)` and call `wp_update_attachment_metadata($id, wp_get_attachment_metadata($id))` per ID via `wp eval-file` (this re-fires the exact same hook a real upload triggers, safely, for existing media). 10. Verify: `mc --insecure ls warrior//wp-content/uploads/` shows growing object count; front-end pages load images from `https://s3.palmasolutions.net//...`. Don't flip `remove-local-file` to true until spot-checked. ## OPEN BUG (found 2026-08-22, NOT fixed): public endpoint breaks SigV4 Every authenticated S3 API request (PUT, even a bodyless LIST) through `https://s3.palmasolutions.net` (Caddy on madri → warrior) fails with `SignatureDoesNotMatch`, for ANY valid MinIO credential pair — confirmed this is not a plugin bug, not a credentials bug, and not a jail-network bug: - Same exact credentials succeed instantly hitting warrior directly (`https://100.78.33.122:9000` or `https://62.210.214.171:9000`, bypassing Caddy entirely). - `storage.palmasolutions.net` (Caddy on the same madri host → stray's MinIO, over **plain HTTP** to the backend) signs correctly — so this isn't a generic "Caddy can't front MinIO" problem. - Two plausible fixes were tried on `infra/madri/Caddyfile`'s `s3.palmasolutions.net` block and **both disproven by direct retest**: removing `encode gzip` (no effect — kept removed anyway, no downside) and forcing `transport http { versions 1.1 }` to rule out Caddy negotiating HTTP/2 to the HTTPS backend via ALPN (no effect either — kept anyway as reasonable hardening, but it is NOT the fix). - `mc admin admin trace -v warrior` failed to connect during this investigation ("unexpected end of JSON input") — packet-level capture (tcpdump/mitmproxy comparing the raw bytes MinIO receives via each path) is the logical next diagnostic step, not more guessing at Caddy directives. **Practical impact**: no WordPress/AS3CF-style plugin (or any other SigV4 client) can currently write to ANY bucket through `s3.palmasolutions.net` — only direct-to-warrior access works. This almost certainly means **easycut.es's own live AS3CF uploads are currently failing the same way** (its wp-config point at this exact endpoint) — worth a dedicated check, since that site has a prior security incident history and any silent failure there deserves attention, not just this new site's setup. Until this is root-caused, treat this skill's "Rollout steps" as correct-but-blocked at step 8 for any NEW site — the config is right, the endpoint isn't currently usable. ## Gotchas - Finding the DB socket on panel-managed Linux hosts (cabrera-style BTPanel/aaPanel installs) is not obvious — `wp-cli` may report "missing MySQL extension" and the default `/run/mysqld/mysqld.sock` path is usually wrong. Check `/tmp/mysql.sock` first before assuming MySQL is down. - Bastille jails (funky, gringo, phreak) don't have this problem — wp-cli works normally inside `jexec wp --allow-root --path= ...`. - Confirm the site's own images aren't already coming from an unrelated external CDN before assuming local uploads are the bottleneck (sateulera-dev's *product* images are CloudFront-hosted already — this offload only applies to the ~950 theme/gallery/local files actually on-disk).