mirror of
https://github.com/Joulenap/joulenap.git
synced 2026-08-11 13:21:43 +02:00
feat(wizard): name a backup server's token after its datastore
A backup server serving two datastores is two devices, which the duplicate guard deliberately allows -- but both wanted a token called joulenap, so setting up the second one deleted and recreated the first one's token. The first device was left holding a dead secret, and re-entering the new one did not repair it: deleting a token also drops its ACL entries, and provisioning re-grants only /datastore/<its own datastore>, so the other datastore stayed locked out until root ran acl update by hand. The product supported a configuration its own wizard could not provision. Tokens on a backup server are now named joulenap-<datastore>, sanitised to the character set PBS accepts and falling back to the bare prefix if nothing survives. The two never meet, and each keeps the narrow per-datastore grant rather than widening to /datastore. A Proxmox host is a single device and cannot collide with itself, so its token stays plain joulenap. The name is derived rather than exposed: a field would only invite tokens Joulenap later fails to find. Tokens already in use are untouched. The conflict dialog no longer claims the name is "joulenap", since on a backup server it is not. A device card also stops reporting "Connected - API OK" for what is a one second TCP connect to the API port. The authenticated call behind it is made and its failure discarded, so a server whose credential had been revoked advertised itself as healthy indefinitely, with cached usage figures beside it to match. The label now reads "Reachable", which is what is actually checked; the Test button, which surfaces the same call's error, owns the API verdict. Changing the underlying field was rejected: it is a published contract, both in the dashboard payload and as joulenap_pbs_online, documented as answering on the API port. Documented in the architecture, the wizard guide and the example config, including that replacing a token clears its permissions -- so a hand-made setup where one token served several datastores needs re-granting.
This commit is contained in:
@@ -20,6 +20,12 @@ guest* reads "never" for every guest and the converted routes read "never run"
|
||||
runs still listed underneath. Nothing has been lost: the caches fill in again per guest as runs
|
||||
happen, and the first run of each route restores its badge.
|
||||
|
||||
**Two limitations worth knowing before you configure.** Joulenap reads the **root namespace** of a
|
||||
backup datastore, so if your Proxmox storage entry writes into a PBS namespace the backups, the
|
||||
retention and the garbage collection all work, but *Last backup per guest* reads "never" for those
|
||||
guests. And two datastores on one backup server are two devices with independent power leases, so
|
||||
consecutive routes across them can sleep and wake that machine once more than strictly necessary.
|
||||
|
||||
### Added
|
||||
|
||||
- **Routes.** A route is one scheduled flow of backup data between devices: sources, a target, its
|
||||
@@ -51,6 +57,13 @@ happen, and the first run of each route restores its badge.
|
||||
walks connection, wake-up (with a Test button that sends a real magic packet before you find out
|
||||
at 04:00 that Wake-on-LAN was never armed) and power-off. Both work from pasted API tokens, or
|
||||
provision everything themselves from a root login used once and never stored.
|
||||
- **A backup server's API token is named after its datastore** — `joulenap-backup`,
|
||||
`joulenap-offsite` — so one machine serving several datastores gets one token per device instead
|
||||
of the setups fighting over a single name. Deleting an API token also drops the permissions
|
||||
granted to it, so a shared name would have meant configuring the second datastore left the first
|
||||
one both locked out and unrepairable by re-entering its secret. A Proxmox host is one device and
|
||||
keeps the plain `joulenap`. Tokens already in use are untouched; the name only applies to ones
|
||||
Joulenap creates from now on.
|
||||
- **Ad-hoc maintenance per backup server.** Run a garbage collection or a verification on one box
|
||||
from the homepage, without a route. It queues and reports like any other run; only the route
|
||||
column is empty.
|
||||
|
||||
@@ -15,6 +15,7 @@ from pydantic import BaseModel, Field
|
||||
from .. import paths
|
||||
from ..connectors import net
|
||||
from ..connectors.errors import ConnectorError, TokenExistsError
|
||||
from ..connectors.provision import pbs_token_name
|
||||
from ..core import wizard
|
||||
from .deps import require_auth
|
||||
|
||||
@@ -122,7 +123,9 @@ class PbsProvisionRequest(BaseModel):
|
||||
username: str = "root@pam"
|
||||
password: str = Field(min_length=1)
|
||||
datastore: str = Field(min_length=1)
|
||||
token_name: str = "joulenap"
|
||||
#: Left unset by the UI: the name is derived from the datastore, so two devices on one
|
||||
#: backup server never contend for it. Still overridable for anyone driving the API.
|
||||
token_name: str | None = None
|
||||
fingerprint: str = ""
|
||||
#: The user's answer to the 409 this endpoint raises when ``token_name`` is taken.
|
||||
replace_token: bool = False
|
||||
@@ -138,7 +141,7 @@ def pbs_provision(body: PbsProvisionRequest) -> dict[str, Any]:
|
||||
username=body.username,
|
||||
password=body.password,
|
||||
datastore=body.datastore,
|
||||
token_name=body.token_name,
|
||||
token_name=body.token_name or pbs_token_name(body.datastore),
|
||||
fingerprint=body.fingerprint,
|
||||
replace_token=body.replace_token,
|
||||
)
|
||||
|
||||
@@ -11,6 +11,7 @@ privileges differ, so the shared logic sits in a base class.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import ssl
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
@@ -42,6 +43,31 @@ PBS_SYSTEM_ROLE = "Audit"
|
||||
# has to be re-provisioned with root, or granted by hand on its console.
|
||||
PBS_REMOTE_ROLES = ("RemoteAdmin", "RemoteSyncPushOperator")
|
||||
|
||||
#: Default token name. A PVE device is one host, so it can never collide with itself.
|
||||
TOKEN_NAME = "joulenap"
|
||||
|
||||
#: PBS token ids accept letters, digits, ``-``, ``_`` and ``.``; anything else is replaced.
|
||||
_TOKEN_UNSAFE = re.compile(r"[^A-Za-z0-9_.-]+")
|
||||
|
||||
|
||||
def pbs_token_name(datastore: str, prefix: str = TOKEN_NAME) -> str:
|
||||
"""The token name for a backup-server device, qualified by its datastore.
|
||||
|
||||
A PBS device is a *(host, datastore)* pair, so one machine can legitimately hold two
|
||||
devices — which the duplicate guard deliberately allows. Naming both tokens ``joulenap``
|
||||
made the second one's setup delete and recreate the first one's token: the first device
|
||||
was left with a dead secret *and*, because deleting a token drops its ACL entries while
|
||||
provisioning only re-grants ``/datastore/<its own datastore>``, no way back in short of a
|
||||
root session. Qualifying the name means the two never meet.
|
||||
|
||||
Sanitised rather than interpolated raw: PBS restricts the character set for token ids, and
|
||||
a datastore whose name survives none of it falls back to the bare prefix rather than
|
||||
producing something the server will reject.
|
||||
"""
|
||||
slug = _TOKEN_UNSAFE.sub("-", datastore.strip()).strip("-._")
|
||||
return f"{prefix}-{slug}" if slug else prefix
|
||||
|
||||
|
||||
_WRITE_METHODS = frozenset({"POST", "PUT", "DELETE"})
|
||||
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.connectors.errors import ApiError, TokenExistsError
|
||||
from app.connectors.provision import pbs_token_name
|
||||
from app.core import wizard
|
||||
from app.main import create_app
|
||||
|
||||
@@ -90,6 +91,63 @@ def test_replace_token_is_forwarded(client, monkeypatch):
|
||||
assert captured["replace_token"] is True
|
||||
|
||||
|
||||
def test_the_pbs_token_name_is_derived_from_the_datastore(client, monkeypatch):
|
||||
"""Two devices on one backup server must not contend for a token name: naming both
|
||||
``joulenap`` meant provisioning the second deleted the first one's token."""
|
||||
captured = {}
|
||||
|
||||
def fake_provision(**kwargs):
|
||||
captured.update(kwargs)
|
||||
return {"id": "root@pam!joulenap-lab", "secret": "s"}
|
||||
|
||||
monkeypatch.setattr(wizard, "pbs_provision", fake_provision)
|
||||
r = client.post(
|
||||
"/api/wizard/pbs/provision",
|
||||
json={"host": "pbs.local", "password": "pw", "datastore": "lab"},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
assert captured["token_name"] == "joulenap-lab"
|
||||
|
||||
|
||||
def test_an_explicit_token_name_still_wins(client, monkeypatch):
|
||||
"""The field stays honoured for anyone driving the API directly."""
|
||||
captured = {}
|
||||
|
||||
def fake_provision(**kwargs):
|
||||
captured.update(kwargs)
|
||||
return {"id": "root@pam!mine", "secret": "s"}
|
||||
|
||||
monkeypatch.setattr(wizard, "pbs_provision", fake_provision)
|
||||
r = client.post(
|
||||
"/api/wizard/pbs/provision",
|
||||
json={
|
||||
"host": "pbs.local",
|
||||
"password": "pw",
|
||||
"datastore": "lab",
|
||||
"token_name": "mine",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
assert captured["token_name"] == "mine"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("datastore", "expected"),
|
||||
[
|
||||
("lab", "joulenap-lab"),
|
||||
("backup_2", "joulenap-backup_2"),
|
||||
("dot.name", "joulenap-dot.name"),
|
||||
(" spaced ", "joulenap-spaced"),
|
||||
("a b/c", "joulenap-a-b-c"),
|
||||
# Nothing survives sanitising, so fall back rather than send PBS a name it rejects.
|
||||
("///", "joulenap"),
|
||||
("", "joulenap"),
|
||||
],
|
||||
)
|
||||
def test_token_name_sanitising(datastore, expected):
|
||||
assert pbs_token_name(datastore) == expected
|
||||
|
||||
|
||||
def test_pbs_check_passes_through(client, monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
wizard, "pbs_check", lambda **_k: {"reachable": True, "fingerprint": "AA:BB"}
|
||||
|
||||
+7
-1
@@ -77,6 +77,10 @@ pbss: []
|
||||
# host: 192.168.1.20 # or pbs.lan
|
||||
# port: 8007
|
||||
# datastore: backup
|
||||
# # Joulenap reads the datastore's ROOT NAMESPACE. If the Proxmox storage entry writes
|
||||
# # into a PBS namespace, backups, retention and GC all work normally, but "Last backup
|
||||
# # per guest" reads "never" for those guests — Joulenap looks in the root and finds
|
||||
# # nothing. One backup server can hold several datastores: add each as its own device.
|
||||
# fingerprint: "" # PBS dashboard > Show Fingerprint, e.g. "aa:bb:cc:..."
|
||||
# # Leave this empty and the connection to this PBS is NOT pinned and NOT validated —
|
||||
# # its API token then travels over TLS nothing has checked. The wizard fills it in.
|
||||
@@ -85,7 +89,9 @@ pbss: []
|
||||
# # sync route also needs RemoteAdmin AND RemoteSyncPushOperator on /remote — PBS refuses
|
||||
# # ACL writes from a token, so those must be granted on the box itself or by the wizard's
|
||||
# # root mode (see docs/CONFIG-WIZARD.md).
|
||||
# api_token_id: "root@pam!joulenap"
|
||||
# # The wizard names a backup server's token after its datastore, so a second datastore on
|
||||
# # the same machine gets its own instead of replacing this one.
|
||||
# api_token_id: "root@pam!joulenap-backup"
|
||||
# api_token_secret: ""
|
||||
# # Set to false for an always-on PBS (a VM, or a cloud-hosted one): Joulenap then only
|
||||
# # schedules routes onto it and never touches its power, so everything below is unused.
|
||||
|
||||
@@ -40,6 +40,8 @@ The **kind** follows from which devices the route names:
|
||||
|
||||
Guests are selected **per source** (`sources[].guests`), because vmids collide between PVEs. Mode is `all` or `include`; in `include` mode a newly created guest is *not* picked up automatically.
|
||||
|
||||
The per-guest last-backup cache is filled by listing the target datastore's snapshots, and that listing covers the datastore's **root namespace only** — no `ns` parameter is sent, and a PBS namespace is configured on the Proxmox storage entry, where Joulenap never sees it. A namespaced setup therefore backs up, prunes and collects garbage correctly while every one of its guests reads *never backed up*. PBS groups are `ct/<vmid>` / `vm/<vmid>` with no record of which host wrote them, so two PVEs sharing a datastore and a vmid also share a group and prune each other's snapshots — use non-overlapping vmid ranges across hosts.
|
||||
|
||||
A route's `schedule` is a time plus seven weekday flags. `schedule.cron` is an escape hatch for anything richer (day-of-month, steps, ranges) and **wins over `time`/`days`** when set; the UI then shows it read-only.
|
||||
|
||||
`options` carries the per-route knobs: `mode` / `bwlimit` / `min_free_percent` (backup only, they are vzdump's), `gc` and `verify_after` (run on the target after the data lands), and `reverify_days` for a verify route. `retention` is vzdump's `prune-backups`, per route.
|
||||
@@ -70,6 +72,8 @@ Only the last of those is worth a warning; the other three are the correct outco
|
||||
|
||||
`managed_power: false` describes an always-on PBS. The lease is the single place that knows: acquiring degrades to a reachability check and releasing does nothing.
|
||||
|
||||
**The lease is keyed per device, not per machine.** A backup server serving two datastores is two devices, so each holds its own lease and neither knows about the other: a run that finishes with the first can power the machine off while a queued run on the second still wants it, and that run then wakes it again. Runs are serialised by the single-run lock, so this costs one extra sleep/wake cycle rather than correctness — but on one physical box, prefer a single datastore, or expect the extra cycle.
|
||||
|
||||
|
||||
## What each kind does
|
||||
|
||||
@@ -221,6 +225,6 @@ A deliberately separate, additive-only contract for third-party widgets, with ma
|
||||
Per device — each PVE and each PBS gets its own token.
|
||||
|
||||
- **PVE token**: `VM.Audit` (list guests) + `VM.Backup` + `Datastore.Audit` + `Datastore.AllocateSpace` **and `Datastore.Allocate`** on the PBS-backed storage (the last is required for vzdump's retention/prune, which deletes old backups). Root-mode setup creates a `Joulenap` role with exactly these privileges (`connectors/provision.py`).
|
||||
- **PBS token**: `DatastoreAdmin` on the datastore (status, GC, verify) plus `Audit` on `/system` (read-only node CPU/RAM/network for the dashboard). PBS has no API to create custom roles, so root-mode setup grants these built-ins scoped by path.
|
||||
- **PBS token**: `DatastoreAdmin` on the datastore (status, GC, verify) plus `Audit` on `/system` (read-only node CPU/RAM/network for the dashboard). PBS has no API to create custom roles, so root-mode setup grants these built-ins scoped by path. The token is named **`joulenap-<datastore>`**: a device is a *(host, datastore)* pair, so one machine can hold two, and a shared name would mean provisioning the second deleted and recreated the first one's token. Deleting a token also drops its ACL entries, so that would have left the first device unable to connect *and* unable to be repaired by re-entering a secret.
|
||||
- **PBS token, additionally, for sync routes**: `RemoteAdmin` **and** `RemoteSyncPushOperator` on `/remote`, so Joulenap can create the remote and the sync job. PBS refuses ACL writes from a token, so these can only be granted from a root login — the wizard does it while it still holds the root ticket, and a box set up before 1.0 gets them from the device editor's **Grant sync permissions** action (`POST /api/wizard/pbs/grant-sync`), which asks for root once and stores nothing. See [`CONFIG-WIZARD.md`](CONFIG-WIZARD.md#sync-routes-need-one-extra-grant).
|
||||
- **SSH to a PBS**: one dedicated key, shared by every managed box, ideally installed with a forced command that only allows `poweroff`.
|
||||
|
||||
+16
-5
@@ -53,17 +53,28 @@ For each field: **auto** = discovered/derived, **manual** = entered.
|
||||
Everything above is still discovered with a read-only token; only these become manual:
|
||||
|
||||
- **On the PVE**: create an API token whose role has `VM.Audit, VM.Backup, Datastore.Audit, Datastore.AllocateSpace, Datastore.Allocate` (the last is required for vzdump's retention/prune, which deletes old backups); copy the secret.
|
||||
- **On the PBS**: create an API token with `DatastoreAdmin` on the datastore (status, GC, verify) **and `Audit` on `/system`** (node CPU/RAM for the dashboard); copy the secret. Add the `/remote` roles below if this box will take part in a sync route.
|
||||
- **On the PBS**: create an API token with `DatastoreAdmin` on the datastore (status, GC, verify) **and `Audit` on `/system`** (node CPU/RAM for the dashboard); copy the secret. Name it per datastore — Joulenap's own wizard uses `joulenap-<datastore>` — so a second datastore on the same box can have its own. Add the `/remote` roles below if this box will take part in a sync route.
|
||||
- **On the PBS**: install Joulenap's generated SSH public key into `/root/.ssh/authorized_keys`.
|
||||
- **In Joulenap**: paste both tokens, confirm the key is installed, click "Detect MAC".
|
||||
|
||||
> **Tighter PBS privileges (optional):** root mode grants the built-in `DatastoreAdmin`, because PBS cannot create custom roles over the API. For a truly minimal token, create a role on the PBS host once and bind a token to it, then paste that token:
|
||||
> ```sh
|
||||
> proxmox-backup-manager role create Joulenap --privs "Datastore.Audit,Datastore.Modify"
|
||||
> proxmox-backup-manager user generate-token root@pam joulenap
|
||||
> proxmox-backup-manager acl update /datastore/<datastore> Joulenap --auth-id 'root@pam!joulenap'
|
||||
> proxmox-backup-manager user generate-token root@pam joulenap-<datastore>
|
||||
> proxmox-backup-manager acl update /datastore/<datastore> Joulenap --auth-id 'root@pam!joulenap-<datastore>'
|
||||
> ```
|
||||
|
||||
> **Token names on a backup server carry the datastore** — `joulenap-backup`, `joulenap-offsite`.
|
||||
> A backup server can hold several datastores, and each is its own device in Joulenap; naming both
|
||||
> tokens the same would mean setting up the second one deleted and recreated the first one's token,
|
||||
> leaving that device unable to connect. A Proxmox host is a single device, so its token stays
|
||||
> plain `joulenap`.
|
||||
>
|
||||
> **Replacing a token also clears the permissions granted to it.** If you ever confirm a
|
||||
> replacement, re-entering the new secret is not always enough: provisioning re-grants only
|
||||
> `/datastore/<that device's datastore>`, so a hand-made setup where one token served several
|
||||
> datastores needs the `acl update` above re-running as root for the others.
|
||||
|
||||
|
||||
## Adding a box you already have
|
||||
|
||||
@@ -128,8 +139,8 @@ A **sync** route makes Joulenap create a *remote* and a *sync job* on one of the
|
||||
The equivalent on the PBS itself, if you would rather not hand the password over:
|
||||
|
||||
```sh
|
||||
proxmox-backup-manager acl update /remote RemoteAdmin --auth-id 'root@pam!joulenap'
|
||||
proxmox-backup-manager acl update /remote RemoteSyncPushOperator --auth-id 'root@pam!joulenap'
|
||||
proxmox-backup-manager acl update /remote RemoteAdmin --auth-id 'root@pam!joulenap-<datastore>'
|
||||
proxmox-backup-manager acl update /remote RemoteSyncPushOperator --auth-id 'root@pam!joulenap-<datastore>'
|
||||
```
|
||||
|
||||
Both roles are needed: `RemoteAdmin` alone does not cover a *push* sync.
|
||||
|
||||
@@ -307,7 +307,7 @@
|
||||
},
|
||||
"tokenExists": {
|
||||
"title": "That token name is already taken",
|
||||
"message": "An API token named “joulenap” already exists on this server. Its secret is only shown when it is created, so Joulenap cannot reuse it — it can only replace it. Anything else still using that token stops working immediately, which on a Proxmox host usually means the backup-server storage entry, and every backup through it. Replace it only if you know nothing else uses it.",
|
||||
"message": "An API token with the name Joulenap uses already exists on this server. Its secret is only shown when it is created, so Joulenap cannot reuse it — it can only replace it. Anything else still using that token stops working immediately, which on a Proxmox host usually means the backup-server storage entry, and every backup through it. Replace it only if you know nothing else uses it.",
|
||||
"confirm": "Replace the token",
|
||||
"usedByDevice": "“{{id}}” in Joulenap already authenticates to this server — replacing the token breaks it.",
|
||||
"usedByStorage": "Storage “{{storage}}” on “{{pve}}” authenticates to this server with it, so backups through that storage would start failing."
|
||||
@@ -534,7 +534,7 @@
|
||||
"alwaysOn": "Always on"
|
||||
},
|
||||
"state": {
|
||||
"connected": "Connected · API OK",
|
||||
"connected": "Reachable",
|
||||
"sleeping": "Sleeping ⏾",
|
||||
"offline": "Offline"
|
||||
},
|
||||
|
||||
@@ -307,7 +307,7 @@
|
||||
},
|
||||
"tokenExists": {
|
||||
"title": "Quel nome di token è già in uso",
|
||||
"message": "Su questo server esiste già un token API chiamato “joulenap”. Il suo segreto viene mostrato solo alla creazione, quindi Joulenap non può riutilizzarlo: può solo sostituirlo. Tutto ciò che usa ancora quel token smette di funzionare all’istante — su un host Proxmox di solito è la voce di storage del backup server, e quindi tutti i backup che ci passano. Sostituiscilo solo se sei sicuro che nessun altro lo usi.",
|
||||
"message": "Su questo server esiste già un token API con il nome che Joulenap usa. Il suo segreto viene mostrato solo alla creazione, quindi Joulenap non può riutilizzarlo: può solo sostituirlo. Tutto ciò che usa ancora quel token smette di funzionare all’istante — su un host Proxmox di solito è la voce di storage del backup server, e quindi tutti i backup che ci passano. Sostituiscilo solo se sei sicuro che nessun altro lo usi.",
|
||||
"confirm": "Sostituisci il token",
|
||||
"usedByDevice": "«{{id}}» in Joulenap si autentica già su questo server: sostituire il token lo rompe.",
|
||||
"usedByStorage": "Lo storage «{{storage}}» su «{{pve}}» si autentica su questo server con quel token, quindi i backup che ci passano inizierebbero a fallire."
|
||||
@@ -534,7 +534,7 @@
|
||||
"alwaysOn": "Sempre acceso"
|
||||
},
|
||||
"state": {
|
||||
"connected": "Connesso · API OK",
|
||||
"connected": "Raggiungibile",
|
||||
"sleeping": "In sospensione ⏾",
|
||||
"offline": "Non raggiungibile"
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user