External/spec-kit
1
0
Fork 0
mirror of https://github.com/github/spec-kit.git synced 2026-08-15 14:02:04 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat(presets): resolve constitution templates at command time (#3984)

Browse Source 
* feat(presets): resolve constitutions at command time

Gate install-time constitution materialization behind the constitution-sync preset while preserving one-time init seeding and authored-file safeguards.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7dbce70f-80c6-4e14-a30d-78cb358bcb84

* fix(presets): emit composed template content

Add a machine-readable preset resolve mode backed by PresetResolver.resolve_content and require the constitution command to consume it.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7dbce70f-80c6-4e14-a30d-78cb358bcb84

* fix(presets): unify runtime template composition

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7dbce70f-80c6-4e14-a30d-78cb358bcb84

* fix(presets): secure runtime template resolution

Align runtime resolution across script variants, validate registry path components, and honor canonical extension ordering and convention paths.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(presets): align runtime priority semantics

Normalize and tie-break preset priorities consistently across script variants, and preserve template bytes when Python materializes generated files.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(presets): stop at effective template base

Avoid parsing irrelevant lower layers once resolution reaches a replace base, and decode raw bytes so Python preserves source line endings.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(presets): align extension template resolution

Support root-level extension templates across runtime resolvers, fail safely when Bash cannot parse an extension registry, and validate requested templates in every prerequisite output mode.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(presets): resolve dotted command identifiers

Route safe dotted names through command resolution, correct traversal coverage, and make Windows CI text decoding explicit.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Avoid orphan feature directories on template errors

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Align malformed preset manifest handling

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c

* Complete runtime resolver parity

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c

* Fail closed on resolver input errors

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c

* Force UTF-8 and full manifest validation

Force UTF-8 decoding for registry and manifest reads in the Bash and
PowerShell embedded-Python parsers so resolution no longer depends on the
process locale, and validate every manifest template entry's required
fields, type, and strategy consistent with the canonical PresetManifest.

Assisted-by: GitHub Copilot (model: claude-opus-4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c

* Fail closed on empty manifests and corrupt registries

Reject manifests missing the provides/templates sections or declaring an
empty template list in all three runtime resolvers, matching the canonical
PresetManifest which treats those as invalid instead of silently degrading a
composing layer to a convention `replace` lookup.

Make a corrupt or unreadable extension registry fail closed in Bash,
PowerShell, and Python instead of swallowing the error and treating every
on-disk extension directory as unregistered-and-enabled, which could activate
a disabled extension.

Read the preset and extension registries as explicit UTF-8 in the PowerShell
resolver so priority/enabled-state decoding no longer depends on the process
code page under Windows PowerShell 5.1.

Assisted-by: GitHub Copilot (model: claude-opus-4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c

* fix(presets): fail closed when extension registry is not a regular file

The Bash and Python resolvers used is_file()/`-f` to gate reading the
extension `.registry`, which returns false for a directory or a broken
symlink at that path. In those cases the resolvers treated the registry
as absent and scanned every on-disk extension directory as unregistered
and enabled — a fail-open path. Detect any filesystem entry at the
registry path (including broken symlinks) and reject unless it is a
readable regular file. PowerShell now rejects a non-leaf entry explicitly
for parity. Adds directory- and broken-symlink parity regressions.

Assisted-by: GitHub Copilot (model: claude-opus-4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c

* fix(presets): fail closed on corrupt registry in canonical resolver and PowerShell

Two remaining fail-open paths for an invalid extension registry:

- The canonical PresetResolver enumerated extensions through
  ExtensionRegistry, whose _load() normalizes a corrupt or unreadable
  registry to an empty mapping. The directory scan then admitted every
  on-disk extension directory as unregistered-and-enabled, so a corrupt
  registry could still supply constitution content at init and through
  constitution-sync materialization. Add a non-invasive is_corrupt()
  probe (recovery behavior for install/enable/disable is unchanged) and
  raise from _get_all_extensions_by_priority() when the registry exists
  but is invalid. _load() now also recovers from OSError/UnicodeDecodeError
  so a directory or unreadable registry no longer crashes construction.

- The PowerShell resolver gated the registry read with Test-Path, which
  returns false for a dangling symlink on Windows, letting a broken
  .registry symlink bypass the guard and enable every on-disk extension.
  Detect the entry via directory enumeration (which observes a broken
  symlink) and reject it unless it is a readable regular file.

Adds canonical corrupt/directory-registry regressions and extends the
broken-symlink parity test to PowerShell.

Assisted-by: GitHub Copilot (model: claude-opus-4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c

* fix(presets): detect dangling registry symlink in ExtensionRegistry.is_corrupt

is_corrupt() gated on Path.exists(), which follows symlinks and returns
False for a dangling .registry symlink — so the canonical PresetResolver
treated it as an absent registry and fell back to scanning every on-disk
extension directory as unregistered-and-enabled, reopening the fail-open
path this guard closes. Detect lexical existence with os.path.lexists and
require a regular file before parsing, so a broken symlink (or directory)
is reported corrupt and resolution fails closed. Adds a canonical
broken-symlink regression alongside the directory case.

Assisted-by: GitHub Copilot (model: claude-opus-4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7dbce70f-80c6-4e14-a30d-78cb358bcb84
Copilot-Session: 3158e06f-95df-4e3a-843f-f159a35aa30c
This commit is contained in:
Manfred Riem
2026-08-10 10:22:31 -05:00
committed by GitHub
co-authored by Copilot App
parent 1a60d1b6b8
commit 16cfab7724
45 changed files with 2721 additions and 361 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/integrations/test_integration_base_markdown.py
+2 -2
View File
@@ -242,11 +242,11 @@ class MarkdownIntegrationTests:
if script_variant == "sh":
for name in ["check-prerequisites.sh", "common.sh", "create-new-feature.sh",
"setup-plan.sh", "setup-tasks.sh"]:
"resolve-template.sh", "setup-plan.sh", "setup-tasks.sh"]:
files.append(f".specify/scripts/bash/{name}")
else:
for name in ["check-prerequisites.ps1", "common.ps1", "create-new-feature.ps1",
"setup-plan.ps1", "setup-tasks.ps1"]:
"resolve-template.ps1", "setup-plan.ps1", "setup-tasks.ps1"]:
files.append(f".specify/scripts/powershell/{name}")
for name in ["checklist-template.md",
tests/integrations/test_integration_base_skills.py
+2
View File
@@ -494,6 +494,7 @@ class SkillsIntegrationTests:
".specify/scripts/bash/check-prerequisites.sh",
".specify/scripts/bash/common.sh",
".specify/scripts/bash/create-new-feature.sh",
".specify/scripts/bash/resolve-template.sh",
".specify/scripts/bash/setup-plan.sh",
".specify/scripts/bash/setup-tasks.sh",
]
@@ -502,6 +503,7 @@ class SkillsIntegrationTests:
".specify/scripts/powershell/check-prerequisites.ps1",
".specify/scripts/powershell/common.ps1",
".specify/scripts/powershell/create-new-feature.ps1",
".specify/scripts/powershell/resolve-template.ps1",
".specify/scripts/powershell/setup-plan.ps1",
".specify/scripts/powershell/setup-tasks.ps1",
]
tests/integrations/test_integration_base_toml.py
+2
View File
@@ -495,6 +495,7 @@ class TomlIntegrationTests:
"check-prerequisites.sh",
"common.sh",
"create-new-feature.sh",
"resolve-template.sh",
"setup-plan.sh",
"setup-tasks.sh",
]:
@@ -504,6 +505,7 @@ class TomlIntegrationTests:
"check-prerequisites.ps1",
"common.ps1",
"create-new-feature.ps1",
"resolve-template.ps1",
"setup-plan.ps1",
"setup-tasks.ps1",
]:
tests/integrations/test_integration_base_yaml.py
+2
View File
@@ -409,6 +409,7 @@ class YamlIntegrationTests:
"check-prerequisites.sh",
"common.sh",
"create-new-feature.sh",
"resolve-template.sh",
"setup-plan.sh",
"setup-tasks.sh",
]:
@@ -418,6 +419,7 @@ class YamlIntegrationTests:
"check-prerequisites.ps1",
"common.ps1",
"create-new-feature.ps1",
"resolve-template.ps1",
"setup-plan.ps1",
"setup-tasks.ps1",
]:
tests/integrations/test_integration_cline.py
+2
View File
@@ -192,6 +192,7 @@ class TestClineIntegration(MarkdownIntegrationTests):
"check-prerequisites.sh",
"common.sh",
"create-new-feature.sh",
"resolve-template.sh",
"setup-plan.sh",
"setup-tasks.sh",
]:
@@ -201,6 +202,7 @@ class TestClineIntegration(MarkdownIntegrationTests):
"check-prerequisites.ps1",
"common.ps1",
"create-new-feature.ps1",
"resolve-template.ps1",
"setup-plan.ps1",
"setup-tasks.ps1",
]:
tests/integrations/test_integration_copilot.py
+3
View File
@@ -278,6 +278,7 @@ class TestCopilotCommandsMode:
".specify/scripts/bash/check-prerequisites.sh",
".specify/scripts/bash/common.sh",
".specify/scripts/bash/create-new-feature.sh",
".specify/scripts/bash/resolve-template.sh",
".specify/scripts/bash/setup-plan.sh",
".specify/scripts/bash/setup-tasks.sh",
".specify/templates/checklist-template.md",
@@ -342,6 +343,7 @@ class TestCopilotCommandsMode:
".specify/scripts/powershell/check-prerequisites.ps1",
".specify/scripts/powershell/common.ps1",
".specify/scripts/powershell/create-new-feature.ps1",
".specify/scripts/powershell/resolve-template.ps1",
".specify/scripts/powershell/setup-plan.ps1",
".specify/scripts/powershell/setup-tasks.ps1",
".specify/templates/checklist-template.md",
@@ -854,6 +856,7 @@ class TestCopilotSkillsMode:
".specify/scripts/bash/check-prerequisites.sh",
".specify/scripts/bash/common.sh",
".specify/scripts/bash/create-new-feature.sh",
".specify/scripts/bash/resolve-template.sh",
".specify/scripts/bash/setup-plan.sh",
".specify/scripts/bash/setup-tasks.sh",
# Templates
tests/integrations/test_integration_generic.py
+2
View File
@@ -348,6 +348,7 @@ class TestGenericIntegration:
".specify/scripts/bash/check-prerequisites.sh",
".specify/scripts/bash/common.sh",
".specify/scripts/bash/create-new-feature.sh",
".specify/scripts/bash/resolve-template.sh",
".specify/scripts/bash/setup-plan.sh",
".specify/scripts/bash/setup-tasks.sh",
".specify/templates/checklist-template.md",
@@ -406,6 +407,7 @@ class TestGenericIntegration:
".specify/scripts/powershell/check-prerequisites.ps1",
".specify/scripts/powershell/common.ps1",
".specify/scripts/powershell/create-new-feature.ps1",
".specify/scripts/powershell/resolve-template.ps1",
".specify/scripts/powershell/setup-plan.ps1",
".specify/scripts/powershell/setup-tasks.ps1",
".specify/templates/checklist-template.md",
tests/parity_helpers.py
+61
View File
@@ -109,6 +109,67 @@ def write_feature_json(
)
def install_composition_stack(
repo: Path, template_name: str, core_content: str
) -> str:
"""Install wrap/prepend/append presets over a core template."""
templates = repo / ".specify" / "templates"
templates.mkdir(parents=True, exist_ok=True)
(templates / f"{template_name}.md").write_text(core_content, encoding="utf-8")
layers = [
("wrap-pack", 1, "wrap", "## Wrapper\n{CORE_TEMPLATE}\n## End\n"),
("prepend-pack", 2, "prepend", "# Prepended\n"),
("append-pack", 3, "append", "# Appended\n"),
]
registry: dict[str, object] = {"presets": {}}
registry_presets = registry["presets"]
assert isinstance(registry_presets, dict)
for preset_id, priority, strategy, content in layers:
preset_dir = repo / ".specify" / "presets" / preset_id
template_dir = preset_dir / "templates"
template_dir.mkdir(parents=True)
(template_dir / f"{template_name}.md").write_text(content, encoding="utf-8")
(preset_dir / "preset.yml").write_text(
"provides:\n"
" templates:\n"
" - type: template\n"
f" name: {template_name}\n"
f" file: templates/{template_name}.md\n"
f" strategy: {strategy}\n",
encoding="utf-8",
)
registry_presets[preset_id] = {
"enabled": True,
"priority": priority,
}
(repo / ".specify" / "presets" / ".registry").write_text(
json.dumps(registry, separators=(",", ":")) + "\n",
encoding="utf-8",
)
appended = "# Appended\n"
prepended = "# Prepended\n"
wrapper = "## Wrapper\n{CORE_TEMPLATE}\n## End\n"
composed = f"{core_content}\n\n{appended}"
composed = f"{prepended}\n\n{composed}"
return wrapper.replace("{CORE_TEMPLATE}", composed)
def break_wrap_layer(repo: Path, template_name: str) -> None:
"""Replace the installed wrap layer with one missing its placeholder."""
(
repo
/ ".specify"
/ "presets"
/ "wrap-pack"
/ "templates"
/ f"{template_name}.md"
).write_text("# Broken wrapper\n", encoding="utf-8")
def normalize_repo_paths(text: str, repo: Path) -> str:
"""Replace the repo path with a placeholder so two-repo runs compare equal."""
repo_paths = sorted({str(repo), str(repo.resolve())}, key=len, reverse=True)
tests/test_check_prerequisites_python_parity.py
+82
View File
@@ -12,6 +12,7 @@ from pathlib import Path
import pytest
from tests.conftest import requires_bash
from tests.parity_helpers import install_composition_stack
PROJECT_ROOT = Path(__file__).resolve().parent.parent
COMMON_SH = PROJECT_ROOT / "scripts" / "bash" / "common.sh"
@@ -136,6 +137,87 @@ def _normalize_help_text(text: str) -> str:
return "\n".join("" if not line.strip() else line for line in normalized.split("\n"))
@requires_bash
@pytest.mark.parametrize("missing", [False, True], ids=["composed", "missing"])
def test_all_variants_resolve_requested_template(
prereq_repo: Path, missing: bool
) -> None:
_write_feature_json(prereq_repo)
feature = prereq_repo / "specs" / "001-my-feature"
feature.mkdir(parents=True)
(feature / "plan.md").write_text("# Plan\n", encoding="utf-8")
template_name = "missing-template" if missing else "checklist-template"
expected = install_composition_stack(
prereq_repo, "checklist-template", "# Checklist\n"
)
results = [
_run(
_bash_cmd(prereq_repo, "--json", "--template", template_name),
prereq_repo,
),
_run(
_py_cmd(prereq_repo, "--json", "--template", template_name),
prereq_repo,
),
]
if HAS_PWSH or _WINDOWS_POWERSHELL:
results.append(
_run(
_ps_cmd(prereq_repo, "-Json", "-Template", template_name),
prereq_repo,
)
)
expected_status = 1 if missing else 0
assert all(result.returncode == expected_status for result in results)
if missing:
assert all(result.stdout == "" for result in results)
else:
assert all(
_json_stdout(result)["TEMPLATE_CONTENT"] == expected
for result in results
)
@requires_bash
@pytest.mark.parametrize("missing", [False, True], ids=["composed", "missing"])
def test_all_variants_validate_requested_template_in_text_mode(
prereq_repo: Path, missing: bool
) -> None:
_write_feature_json(prereq_repo)
feature = prereq_repo / "specs" / "001-my-feature"
feature.mkdir(parents=True)
(feature / "plan.md").write_text("# Plan\n", encoding="utf-8")
template_name = "missing-template" if missing else "checklist-template"
install_composition_stack(
prereq_repo, "checklist-template", "# Checklist\n"
)
results = [
_run(
_bash_cmd(prereq_repo, "--template", template_name),
prereq_repo,
),
_run(
_py_cmd(prereq_repo, "--template", template_name),
prereq_repo,
),
]
if HAS_PWSH or _WINDOWS_POWERSHELL:
results.append(
_run(
_ps_cmd(prereq_repo, "-Template", template_name),
prereq_repo,
)
)
expected_status = 1 if missing else 0
assert all(result.returncode == expected_status for result in results)
if missing:
assert all(result.stdout == "" for result in results)
@requires_bash
@pytest.mark.parametrize(
"args",
tests/test_command_template_py_scripts.py
+1 -1
View File
@@ -79,7 +79,7 @@ def test_template_renders_python_invocation(name: str):
result = IntegrationBase.process_template(content, "agent", "py")
assert "{SCRIPT}" not in result
assert re.search(
r"python3 \.specify/scripts/python/\w+\.py(?: --[\w-]+)*", result
r"python3 \.specify/scripts/python/\w+\.py(?: [\w-]+)*", result
), f"{name} did not render a Python invocation"
tests/test_create_new_feature_python_parity.py
+99 -1
View File
@@ -13,6 +13,8 @@ from tests.conftest import requires_bash
from tests.parity_helpers import (
HAS_POWERSHELL,
bash_cmd,
break_wrap_layer,
install_composition_stack,
install_scripts,
json_stdout,
make_repo,
@@ -382,12 +384,108 @@ def test_python_full_run_matches_bash(repo_pair: tuple[Path, Path]) -> None:
branch = json_stdout(py)["BRANCH_NAME"]
for repo in repo_pair:
spec = repo / "specs" / branch / "spec.md"
assert spec.read_text(encoding="utf-8") == TEMPLATE_BODY
assert spec.read_bytes() == TEMPLATE_BODY.encode("utf-8")
assert (repo_b / ".specify" / "feature.json").read_bytes() == (
repo_a / ".specify" / "feature.json"
).read_bytes()
@requires_bash
def test_all_variants_materialize_composed_spec_template(tmp_path: Path) -> None:
repos = [
_setup_repo(tmp_path, "bash"),
_setup_repo(tmp_path, "powershell"),
_setup_repo(tmp_path, "python"),
]
expected = ""
for current in repos:
expected = install_composition_stack(
current, "spec-template", TEMPLATE_BODY
)
bash = run(
bash_cmd(
repos[0],
SCRIPT,
"--json",
"--number",
"1",
"--short-name",
"composed",
"x",
),
repos[0],
)
py = run(
py_cmd(
repos[2],
SCRIPT,
"--json",
"--number",
"1",
"--short-name",
"composed",
"x",
),
repos[2],
)
results = [bash, py]
checked_repos = [repos[0], repos[2]]
if HAS_POWERSHELL:
results.insert(
1,
run(
ps_cmd(
repos[1],
SCRIPT,
"-Json",
"-Number",
"1",
"-ShortName",
"composed",
"x",
),
repos[1],
),
)
checked_repos.insert(1, repos[1])
assert all(result.returncode == 0 for result in results)
for current in checked_repos:
assert (
current / "specs" / "001-composed" / "spec.md"
).read_text(encoding="utf-8") == expected
@requires_bash
def test_all_variants_fail_for_broken_spec_composition(tmp_path: Path) -> None:
repos = [
_setup_repo(tmp_path, "bash"),
_setup_repo(tmp_path, "powershell"),
_setup_repo(tmp_path, "python"),
]
for current in repos:
install_composition_stack(current, "spec-template", TEMPLATE_BODY)
break_wrap_layer(current, "spec-template")
bash = run(bash_cmd(repos[0], SCRIPT, "--json", "x"), repos[0])
py = run(py_cmd(repos[2], SCRIPT, "--json", "x"), repos[2])
results = [(bash, repos[0]), (py, repos[2])]
if HAS_POWERSHELL:
results.append(
(
run(ps_cmd(repos[1], SCRIPT, "-Json", "x"), repos[1]),
repos[1],
)
)
assert all(result.returncode != 0 for result, _ in results)
assert all(
not (current / "specs" / "001-x").exists()
for _, current in results
)
@requires_bash
def test_python_missing_template_warning_matches_bash(
repo_pair: tuple[Path, Path],
tests/test_presets.py
+205 -33
View File
@@ -1135,6 +1135,40 @@ class TestPresetResolver:
result = resolver.resolve("nonexistent-template")
assert result is None
def test_resolver_ignores_traversing_registry_ids(self, project_dir):
"""Registry IDs cannot escape preset or extension install roots."""
for registry_dir, registry_key, outside_name in (
("presets", "presets", "outside-preset"),
("extensions", "extensions", "outside-extension"),
):
outside = project_dir.parent / outside_name
(outside / "templates").mkdir(parents=True)
(outside / "templates" / "spec-template.md").write_text(
f"# Sensitive {registry_key}\n",
encoding="utf-8",
)
installed = project_dir / ".specify" / registry_dir
installed.mkdir(parents=True, exist_ok=True)
(installed / ".registry").write_text(
json.dumps(
{
registry_key: {
f"../../../{outside_name}": {
"enabled": True,
"priority": 1,
}
}
}
),
encoding="utf-8",
)
content = PresetResolver(project_dir).resolve_content("spec-template")
assert content is not None
assert "Core Spec Template" in content
assert "Sensitive" not in content
def test_resolve_higher_priority_pack_wins(self, project_dir, temp_dir, valid_pack_data):
"""Test that a pack with lower priority number wins over higher number."""
manager = PresetManager(project_dir)
@@ -1447,6 +1481,65 @@ class TestPresetResolver:
result = resolver.resolve("unique-disabled-template")
assert result is None, "Disabled extension should not be picked up as unregistered"
@pytest.mark.parametrize(
"registry_bytes",
[b"{ not valid json", b'{"extensions": []}', b"[]"],
ids=["invalid_json", "non_mapping_extensions", "non_mapping_root"],
)
def test_resolve_fails_closed_on_corrupt_extension_registry(
self, project_dir, registry_bytes
):
"""A corrupt extension registry must fail closed rather than let the
directory scan admit every on-disk extension as enabled."""
extensions_dir = project_dir / ".specify" / "extensions"
ext_templates_dir = extensions_dir / "sneaky-ext" / "templates"
ext_templates_dir.mkdir(parents=True)
(ext_templates_dir / "custom-template.md").write_text(
"# Should not be served\n"
)
(extensions_dir / ".registry").write_bytes(registry_bytes)
resolver = PresetResolver(project_dir)
with pytest.raises(PresetValidationError, match="Invalid extension registry"):
resolver._get_all_extensions_by_priority()
with pytest.raises(PresetValidationError, match="Invalid extension registry"):
resolver.resolve("custom-template")
def test_resolve_fails_closed_when_registry_is_directory(self, project_dir):
"""A directory at the registry path must fail closed, not be treated as
an absent registry that enables every on-disk extension."""
extensions_dir = project_dir / ".specify" / "extensions"
ext_templates_dir = extensions_dir / "sneaky-ext" / "templates"
ext_templates_dir.mkdir(parents=True)
(ext_templates_dir / "custom-template.md").write_text(
"# Should not be served\n"
)
(extensions_dir / ".registry").mkdir()
resolver = PresetResolver(project_dir)
with pytest.raises(PresetValidationError, match="Invalid extension registry"):
resolver.resolve("custom-template")
def test_resolve_fails_closed_when_registry_is_broken_symlink(self, project_dir):
"""A dangling ``.registry`` symlink must fail closed. ``Path.exists()``
follows symlinks and would mistake it for an absent registry, reopening
the fail-open directory scan."""
extensions_dir = project_dir / ".specify" / "extensions"
ext_templates_dir = extensions_dir / "sneaky-ext" / "templates"
ext_templates_dir.mkdir(parents=True)
(ext_templates_dir / "custom-template.md").write_text(
"# Should not be served\n"
)
(extensions_dir / ".registry").symlink_to(
extensions_dir / "does-not-exist"
)
registry = ExtensionRegistry(extensions_dir)
assert registry.is_corrupt()
resolver = PresetResolver(project_dir)
with pytest.raises(PresetValidationError, match="Invalid extension registry"):
resolver.resolve("custom-template")
def test_resolve_pack_over_extension(self, project_dir, pack_dir, temp_dir, valid_pack_data):
"""Test that pack templates take priority over extension templates."""
# Create extension with templates
@@ -3454,6 +3547,9 @@ class TestPresetCatalogMultiCatalog:
SELF_TEST_PRESET_DIR = Path(__file__).parent.parent / "presets" / "self-test"
CONSTITUTION_SYNC_PRESET_DIR = (
Path(__file__).parent.parent / "presets" / "constitution-sync"
)
SELF_TEST_WRAP_WARNING = (
r"Cannot compose command 'speckit\.wrap-test': no base layer\. "
r"Stale command files may remain\."
@@ -3480,6 +3576,11 @@ def install_self_test_preset(manager: PresetManager, speckit_version: str = "0.1
return manager.install_from_directory(SELF_TEST_PRESET_DIR, speckit_version)
def install_constitution_sync_preset(manager: PresetManager) -> PresetManifest:
"""Enable guarded install-time constitution materialization."""
return manager.install_from_directory(CONSTITUTION_SYNC_PRESET_DIR, "0.15.0")
def _make_convention_constitution_preset(temp_dir: Path) -> Path:
"""Create a preset whose constitution is found by convention, not its manifest."""
preset_dir = temp_dir / "convention-constitution"
@@ -3612,6 +3713,7 @@ class TestSelfTestPreset:
(templates_dir / f"{name}.md").write_text(f"# Core {name}\n")
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
manager.remove("self-test")
@@ -3630,6 +3732,7 @@ class TestSelfTestPreset:
(templates_dir / "constitution-template.md").write_text("# Core Constitution\n")
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
memory = project_dir / ".specify" / "memory" / "constitution.md"
edited = memory.read_text() + "\n## Authored amendment\n"
@@ -3713,19 +3816,16 @@ class TestSelfTestPreset:
metadata = manager.registry.get("self-test")
assert metadata["registered_commands"] == {}
def test_self_test_seeds_constitution_when_memory_absent(self, project_dir):
"""Installing a preset seeds memory/constitution.md from its template."""
def test_self_test_does_not_seed_constitution_without_sync(self, project_dir):
"""Installing a preset does not materialize its constitution by default."""
manager = PresetManager(project_dir)
install_self_test_preset(manager)
memory = project_dir / ".specify" / "memory" / "constitution.md"
assert memory.exists(), "constitution.md was not seeded from the preset"
assert "preset:self-test" in memory.read_text(), (
"constitution.md was not seeded from the self-test preset template"
)
assert not memory.exists()
def test_self_test_reseeds_exact_core_constitution(self, project_dir):
"""An unchanged core constitution is re-seeded from the preset template."""
def test_self_test_preserves_generated_constitution_without_sync(self, project_dir):
"""Preset install and removal preserve generated content without the opt-in."""
resolver = PresetResolver(project_dir)
bundled_core = resolver._find_bundled_core(
"constitution-template", "template", ".md"
@@ -3738,10 +3838,19 @@ class TestSelfTestPreset:
manager = PresetManager(project_dir)
install_self_test_preset(manager)
manager.remove("self-test")
content = memory.read_text()
assert "preset:self-test" in content, "placeholder constitution was not re-seeded"
assert "[PROJECT_NAME]" not in content
assert memory.read_bytes() == core
def test_self_test_seeds_constitution_with_sync(self, project_dir):
"""constitution-sync preserves the previous install-time seeding behavior."""
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
memory = project_dir / ".specify" / "memory" / "constitution.md"
assert "preset:self-test" in memory.read_text()
assert "[PROJECT_NAME]" not in memory.read_text()
@pytest.mark.parametrize(
"provenance_content",
@@ -3769,6 +3878,7 @@ class TestSelfTestPreset:
original = memory.read_bytes()
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
assert memory.read_bytes() == original
@@ -3785,6 +3895,7 @@ class TestSelfTestPreset:
memory.write_text(authored)
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
assert memory.read_text() == authored
@@ -3831,7 +3942,9 @@ class TestSelfTestPreset:
)
)
PresetManager(project_dir).install_from_directory(preset_dir, "0.1.5")
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
manager.install_from_directory(preset_dir, "0.1.5")
assert memory.read_text() == authored
assert not (memory.parent / ".constitution-template.json").exists()
@@ -3846,6 +3959,7 @@ class TestSelfTestPreset:
memory.write_text(authored)
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
assert memory.read_text() == authored
@@ -3858,6 +3972,7 @@ class TestSelfTestPreset:
memory.write_text(authored)
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
assert memory.read_text() == authored, "authored constitution was overwritten"
@@ -3915,6 +4030,7 @@ class TestSelfTestPreset:
)
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
manager.install_from_directory(preset_dir, "0.1.5")
memory = project_dir / ".specify" / "memory" / "constitution.md"
@@ -3928,6 +4044,7 @@ class TestSelfTestPreset:
):
"""An unchanged generated constitution follows priority and fallback layers."""
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
preset_dir = temp_dir / "higher-priority"
@@ -3975,6 +4092,7 @@ class TestSelfTestPreset:
):
"""Removing a convention layer rematerializes the remaining resolver layer."""
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
manager.install_from_directory(
_make_convention_constitution_preset(temp_dir), "0.1.5", priority=1
@@ -3996,6 +4114,7 @@ class TestSelfTestPreset:
templates_dir = project_dir / ".specify" / "templates"
(templates_dir / "constitution-template.md").write_text("# Core Constitution\n")
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
manager.install_from_directory(
_make_convention_constitution_preset(temp_dir), "0.1.5"
)
@@ -4013,6 +4132,7 @@ class TestSelfTestPreset:
):
"""Provenance triggers fallback when a custom-path manifest is invalid."""
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
preset_dir = temp_dir / "custom-constitution"
@@ -4073,9 +4193,9 @@ class TestSelfTestPreset:
manager = PresetManager(project_dir)
with pytest.warns(UserWarning, match="symlinked"):
install_self_test_preset(manager)
install_constitution_sync_preset(manager)
assert manager.registry.is_installed("self-test")
assert manager.registry.is_installed("constitution-sync")
assert not (outside / "constitution.md").exists()
def test_constitution_seed_rejects_dangling_destination_symlink(
@@ -4092,9 +4212,9 @@ class TestSelfTestPreset:
manager = PresetManager(project_dir)
with pytest.warns(UserWarning, match="symlinked"):
install_self_test_preset(manager)
install_constitution_sync_preset(manager)
assert manager.registry.is_installed("self-test")
assert manager.registry.is_installed("constitution-sync")
assert not outside.exists()
def test_constitution_materialization_error_is_nonfatal(
@@ -4133,6 +4253,7 @@ class TestSelfTestPreset:
)
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
with pytest.warns(UserWarning, match="Failed to seed constitution"):
manifest = manager.install_from_directory(preset_dir, "0.1.5")
@@ -9786,6 +9907,7 @@ class TestPresetSetPriority:
from specify_cli import app
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
manager.install_from_directory(
_make_convention_constitution_preset(temp_dir), "0.1.5", priority=20
@@ -10031,6 +10153,7 @@ class TestPresetEnableDisable:
from specify_cli import app
manager = PresetManager(project_dir)
install_constitution_sync_preset(manager)
install_self_test_preset(manager)
manager.install_from_directory(
_make_convention_constitution_preset(temp_dir), "0.1.5", priority=1
@@ -10251,6 +10374,29 @@ def test_constitution_commands_guard_against_non_governance_work(command_path):
assert "do not invoke it" in normalized_content or "without invoking it" in normalized_content
def test_core_constitution_command_resolves_template_at_runtime():
"""The core command must consume the composed scaffold on every invocation."""
content = CORE_CONSTITUTION_COMMAND.read_text()
assert "resolve-template.sh constitution-template --json" in content
assert "resolve-template.ps1 constitution-template -Json" in content
assert "resolve_template.py constitution-template --json" in content
assert "parse `TEMPLATE_CONTENT` as the active template" in content
assert "do not continue with only one contributing" in content
assert "Do not write back to any versioned template layer" in content
def test_core_checklist_command_resolves_template_at_runtime():
"""The checklist command must consume the composed scaffold."""
content = (CORE_CONSTITUTION_COMMAND.parent / "checklist.md").read_text(
encoding="utf-8"
)
assert "--template checklist-template" in content
assert "TEMPLATE_CONTENT" in content
assert "Use TEMPLATE_CONTENT as the structural template" in content
class TestLeanPreset:
"""Tests for the lean preset that ships with the repo."""
@@ -12779,10 +12925,10 @@ def test_preset_wrapper_resolves_ghes_asset_when_host_configured(tmp_path, monke
class TestEnsureConstitutionResolverAware:
"""`ensure_constitution_from_template` must resolve through PresetResolver.
The constitution is the only template materialized to a live file rather
than resolved on demand. These tests pin the regression from issue #3272:
a preset-provided ``constitution-template`` must seed memory, while the
core template is used when no preset overrides it.
Init materializes the live constitution once, while later /constitution
runs resolve on demand. These tests pin the regression from issue #3272:
a preset-provided ``constitution-template`` must win during the init seed,
while the core template is used when no preset overrides it.
"""
def _core_constitution(self, project_dir):
@@ -12843,10 +12989,8 @@ class TestEnsureConstitutionResolverAware:
manager = PresetManager(project_dir)
install_self_test_preset(manager)
# Remove the memory file seeded during install to test ensure() in
# isolation; it must re-seed from the preset, not the core template.
memory = project_dir / ".specify" / "memory" / "constitution.md"
memory.unlink()
assert not memory.exists()
ensure_constitution_from_template(project_dir)
@@ -12889,9 +13033,8 @@ class TestEnsureConstitutionResolverAware:
manager = PresetManager(project_dir)
manager.install_from_directory(self._wrap_constitution_preset(temp_dir), "0.1.5")
# Ensure we validate ensure() behavior directly.
memory = project_dir / ".specify" / "memory" / "constitution.md"
memory.unlink()
assert not memory.exists()
ensure_constitution_from_template(project_dir)
content = memory.read_text()
@@ -13260,11 +13403,41 @@ class TestInstalledPresetRichMarkup:
assert result.exit_code == 0, (args, result.output, result.exception)
assert "Broken [/red] tag" in strip_ansi(result.output)
def test_resolve_escapes_template_name(self, project_dir):
"""``preset resolve`` echoes its argument; an unbalanced tag must not crash."""
def test_resolve_rejects_invalid_template_name(self, project_dir):
"""``preset resolve`` rejects names before joining them into paths."""
result = self._invoke(project_dir, ["preset", "resolve", "no[/red]such"])
assert result.exit_code == 1, (result.output, result.exception)
assert "invalid template name" in strip_ansi(result.output)
def test_resolve_rejects_path_traversal(self, project_dir):
"""The resolver rejects traversal before joining names into paths."""
result = self._invoke(
project_dir,
["preset", "resolve", "../../../README"],
)
assert result.exit_code == 1
assert "invalid template name" in strip_ansi(result.output)
def test_resolve_accepts_dotted_command_name(self, project_dir):
"""Documented dotted command identifiers use command resolution."""
result = self._invoke(
project_dir,
["preset", "resolve", "speckit.constitution"],
)
assert result.exit_code == 0, (result.output, result.exception)
assert "no[/red]such" in strip_ansi(result.output)
assert "constitution.md" in strip_ansi(result.output)
def test_resolve_rejects_empty_command_segments(self, project_dir):
"""Dotted command identifiers cannot contain empty path-like segments."""
result = self._invoke(
project_dir,
["preset", "resolve", "speckit..constitution"],
)
assert result.exit_code == 1
assert "invalid template name" in strip_ansi(result.output)
def test_resolve_escapes_layer_path_and_source(self, project_dir):
"""The top-layer path/source lines must render markup literally.
@@ -13358,14 +13531,13 @@ class TestInstalledPresetRichMarkup:
assert "[base]" in output, output
assert "[append]" in output, output
class TestConstitutionSyncPreset:
"""The bundled opt-in ``constitution-sync`` preset re-adds propagation.
"""The bundled opt-in ``constitution-sync`` preset re-adds materialization.
Follow-up to #3790: core ``/constitution`` no longer propagates guidance
into templates. This preset restores that behavior for teams that treat
materialized templates as reviewed artifacts, delivered as a ``wrap`` of
the core command so it stays forward-compatible with core changes.
into templates. Issue #3950 also gates install-time constitution seeding on
this preset. Its command override remains a ``wrap`` of core so it stays
forward-compatible with core changes.
"""
PRESET_DIR = Path(__file__).parent.parent / "presets" / "constitution-sync"
tests/test_resolve_template_python_parity.py
+753
View File
@@ -0,0 +1,753 @@
"""Parity tests for composed runtime template resolution."""
from __future__ import annotations
import json
import os
from pathlib import Path
import pytest
from tests.conftest import requires_bash
from tests.parity_helpers import (
HAS_POWERSHELL,
bash_cmd,
clean_env,
install_composition_stack,
install_scripts,
json_stdout,
make_repo,
ps_cmd,
py_cmd,
run,
)
SCRIPT = "resolve-template"
TEMPLATE = "constitution-template"
def _setup_repo(tmp_path: Path) -> tuple[Path, str]:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
expected = install_composition_stack(repo, TEMPLATE, "# Core\n")
return repo, expected
@requires_bash
def test_all_variants_emit_composed_template_content(tmp_path: Path) -> None:
repo, expected = _setup_repo(tmp_path)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(result.stderr == "" for result in results)
assert all(
json_stdout(result)
== {"TEMPLATE_NAME": TEMPLATE, "TEMPLATE_CONTENT": expected}
for result in results
)
@requires_bash
@pytest.mark.parametrize(
"without_registry,core_content",
[
(True, "# Core\n"),
(False, "# Café ✓\n"),
],
ids=["directory_fallback", "unicode"],
)
def test_all_variants_preserve_composition_parity(
tmp_path: Path, without_registry: bool, core_content: str
) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
expected = install_composition_stack(repo, TEMPLATE, core_content)
if without_registry:
(repo / ".specify" / "presets" / ".registry").unlink()
expected = (
"# Prepended\n\n\n"
"## Wrapper\n"
f"{core_content}\n"
"## End\n\n\n"
"# Appended\n"
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TEMPLATE_CONTENT"] == expected
for result in results
)
@requires_bash
def test_all_variants_read_utf8_registry_under_ascii_locale(
tmp_path: Path,
) -> None:
"""Registry/manifest reads must force UTF-8, not the process locale.
With UTF-8 mode disabled and a C locale, the interpreter's default text
encoding is ASCII. Non-ASCII *metadata* in the registry or a manifest must
still resolve, because the resolvers open those files as UTF-8 explicitly.
Template content stays ASCII so the pure-Python variant can emit it on the
ASCII stdout this configuration forces.
"""
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
expected = install_composition_stack(repo, TEMPLATE, "# Core\n")
# Inject non-ASCII metadata into the preset registry and a manifest so a
# locale-dependent decode would raise instead of resolving cleanly.
registry = repo / ".specify" / "presets" / ".registry"
registry_data = json.loads(registry.read_text(encoding="utf-8"))
registry_data["presets"]["wrap-pack"]["description"] = "Café ✓ wrapper"
registry.write_text(
json.dumps(registry_data, separators=(",", ":")) + "\n",
encoding="utf-8",
)
manifest = repo / ".specify" / "presets" / "wrap-pack" / "preset.yml"
manifest.write_text(
manifest.read_text(encoding="utf-8") + ' description: "Café ✓"\n',
encoding="utf-8",
)
env = clean_env()
# Force the interpreter's default text encoding to ASCII so an unqualified
# open() would fail on the non-ASCII metadata above.
env["PYTHONUTF8"] = "0"
env["PYTHONCOERCECLOCALE"] = "0"
env["LC_ALL"] = "C"
env["LANG"] = "C"
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo, env))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TEMPLATE_CONTENT"] == expected
for result in results
)
@requires_bash
@pytest.mark.parametrize(
"template_name",
["missing-template", "../../../outside"],
ids=["missing", "path_traversal"],
)
def test_all_variants_reject_unresolvable_template(
tmp_path: Path, template_name: str
) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
(repo / "outside.md").write_text("sensitive content\n", encoding="utf-8")
results = [
run(bash_cmd(repo, SCRIPT, template_name, "--json"), repo),
run(py_cmd(repo, SCRIPT, template_name, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, template_name, "-Json"), repo))
assert all(result.returncode == 1 for result in results)
assert all(result.stdout == "" for result in results)
assert all("sensitive content" not in result.stderr for result in results)
@requires_bash
def test_all_variants_ignore_traversing_preset_registry_ids(tmp_path: Path) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
outside = repo.parent / "outside"
outside.mkdir()
(outside / f"{TEMPLATE}.md").write_text("sensitive content\n", encoding="utf-8")
presets = repo / ".specify" / "presets"
presets.mkdir(parents=True)
(presets / ".registry").write_text(
'{"presets":{"../../../outside":{"enabled":true,"priority":1}}}\n',
encoding="utf-8",
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 1 for result in results)
assert all("sensitive content" not in result.stdout for result in results)
@requires_bash
def test_all_variants_support_root_level_preset_convention(tmp_path: Path) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
preset = repo / ".specify" / "presets" / "root-pack"
preset.mkdir(parents=True)
(preset / f"{TEMPLATE}.md").write_text("# Root convention\n", encoding="utf-8")
(repo / ".specify" / "presets" / ".registry").write_text(
'{"presets":{"root-pack":{"enabled":true,"priority":1}}}\n',
encoding="utf-8",
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TEMPLATE_CONTENT"] == "# Root convention\n"
for result in results
)
@requires_bash
def test_all_variants_honor_extension_registry_state_and_priority(
tmp_path: Path,
) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
extensions = repo / ".specify" / "extensions"
for extension_id, content in (
("disabled-ext", "# Disabled\n"),
("low-priority", "# Low priority\n"),
("high-priority", "# High priority\n"),
):
template_dir = extensions / extension_id / "templates"
template_dir.mkdir(parents=True)
(template_dir / f"{TEMPLATE}.md").write_text(content, encoding="utf-8")
(extensions / ".registry").write_text(
'{"extensions":{'
'"disabled-ext":{"enabled":null,"priority":1},'
'"low-priority":{"enabled":true,"priority":20},'
'"high-priority":{"enabled":true,"priority":5}'
"}}\n",
encoding="utf-8",
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TEMPLATE_CONTENT"] == "# High priority\n"
for result in results
)
@requires_bash
def test_all_variants_support_root_level_extension_convention(
tmp_path: Path,
) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
extension = repo / ".specify" / "extensions" / "root-extension"
extension.mkdir(parents=True)
(extension / f"{TEMPLATE}.md").write_text(
"# Root extension\n",
encoding="utf-8",
)
(repo / ".specify" / "extensions" / ".registry").write_text(
'{"extensions":{"root-extension":{"enabled":true,"priority":1}}}\n',
encoding="utf-8",
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TEMPLATE_CONTENT"] == "# Root extension\n"
for result in results
)
@requires_bash
def test_all_variants_treat_extension_registry_ids_case_sensitively(
tmp_path: Path,
) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
extension = repo / ".specify" / "extensions" / "foo" / "templates"
extension.mkdir(parents=True)
(extension / f"{TEMPLATE}.md").write_text(
"# Lowercase extension\n",
encoding="utf-8",
)
(repo / ".specify" / "extensions" / ".registry").write_text(
'{"extensions":{"FOO":{"enabled":true,"priority":1}}}\n',
encoding="utf-8",
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TEMPLATE_CONTENT"] == "# Lowercase extension\n"
for result in results
)
@requires_bash
@pytest.mark.parametrize(
"registry_content",
["{ not valid json", '{"extensions":[]}\n', "[]\n"],
ids=["invalid_json", "non_mapping_extensions", "non_mapping_root"],
)
def test_all_variants_fail_for_malformed_extension_registry(
tmp_path: Path, registry_content: str
) -> None:
"""A corrupt extension registry must fail closed, not silently enable
every on-disk extension directory as unregistered."""
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
extensions = repo / ".specify" / "extensions"
template_dir = extensions / "sneaky-ext" / "templates"
template_dir.mkdir(parents=True)
(template_dir / f"{TEMPLATE}.md").write_text(
"# Should not be served\n", encoding="utf-8"
)
(extensions / ".registry").write_text(registry_content, encoding="utf-8")
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode != 0 for result in results)
assert all(result.stdout == "" for result in results)
assert all(
"Should not be served" not in result.stdout for result in results
)
@requires_bash
def test_all_variants_fail_when_registry_is_a_directory(
tmp_path: Path,
) -> None:
"""A directory at the extension registry path must fail closed, not be
treated as an absent registry that enables every on-disk extension."""
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
extensions = repo / ".specify" / "extensions"
template_dir = extensions / "sneaky-ext" / "templates"
template_dir.mkdir(parents=True)
(template_dir / f"{TEMPLATE}.md").write_text(
"# Should not be served\n", encoding="utf-8"
)
# Create ``.registry`` as a directory rather than a regular file.
(extensions / ".registry").mkdir()
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode != 0 for result in results)
assert all(result.stdout == "" for result in results)
@requires_bash
def test_all_variants_fail_when_registry_is_broken_symlink(
tmp_path: Path,
) -> None:
"""A broken symlink at the extension registry path must fail closed across
Bash, Python, and PowerShell resolvers rather than being treated as absent."""
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
extensions = repo / ".specify" / "extensions"
template_dir = extensions / "sneaky-ext" / "templates"
template_dir.mkdir(parents=True)
(template_dir / f"{TEMPLATE}.md").write_text(
"# Should not be served\n", encoding="utf-8"
)
(extensions / ".registry").symlink_to(extensions / "does-not-exist")
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode != 0 for result in results)
assert all(result.stdout == "" for result in results)
@requires_bash
@pytest.mark.parametrize("base_kind", ["override", "preset"])
def test_all_variants_ignore_malformed_layers_below_replace_base(
tmp_path: Path,
base_kind: str,
) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
expected = "# Winning base\r\nBody\r\n"
presets = repo / ".specify" / "presets"
if base_kind == "override":
override = repo / ".specify" / "templates" / "overrides"
override.mkdir(parents=True)
(override / f"{TEMPLATE}.md").write_bytes(expected.encode("utf-8"))
registry = {"presets": {"broken-pack": {"enabled": True, "priority": 1}}}
else:
winning = presets / "winning-pack" / "templates"
winning.mkdir(parents=True)
(winning / f"{TEMPLATE}.md").write_bytes(expected.encode("utf-8"))
registry = {
"presets": {
"winning-pack": {"enabled": True, "priority": 1},
"broken-pack": {"enabled": True, "priority": 2},
}
}
broken = presets / "broken-pack"
broken.mkdir(parents=True)
(broken / "preset.yml").write_text("provides: [\n", encoding="utf-8")
(presets / ".registry").write_text(
json.dumps(registry, separators=(",", ":")) + "\n",
encoding="utf-8",
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TEMPLATE_CONTENT"] == expected
for result in results
)
@requires_bash
@pytest.mark.parametrize(
("entries", "expected"),
[
(
[
("disabled-pack", {"enabled": False, "priority": 0}),
("numeric-pack", {"enabled": True, "priority": 2}),
("string-pack", {"enabled": True, "priority": "1"}),
],
"# string-pack\n",
),
(
[
("z-pack", {"enabled": True}),
("a-pack", {"enabled": True}),
],
"# a-pack\n",
),
(
[
("float-pack", {"enabled": True, "priority": 5.9}),
("six-pack", {"enabled": True, "priority": 6}),
],
"# float-pack\n",
),
(
[
("a-huge-pack", {"enabled": True, "priority": 2147483648}),
("z-default-pack", {"enabled": True, "priority": "invalid"}),
],
"# z-default-pack\n",
),
(
[
("decimal-string-pack", {"enabled": True, "priority": "5.9"}),
("exponent-string-pack", {"enabled": True, "priority": "1e3"}),
("hex-string-pack", {"enabled": True, "priority": "0x10"}),
("six-pack", {"enabled": True, "priority": 6}),
],
"# six-pack\n",
),
],
ids=[
"mixed_priorities",
"equal_priority_id_tiebreaker",
"float_priority",
"large_integer_priority",
"non_integer_numeric_strings",
],
)
def test_all_variants_normalize_and_tiebreak_preset_priorities(
tmp_path: Path,
entries: list[tuple[str, dict[str, object]]],
expected: str,
) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
presets = repo / ".specify" / "presets"
registry: dict[str, object] = {"presets": {}}
registry_presets = registry["presets"]
assert isinstance(registry_presets, dict)
for preset_id, metadata in entries:
template_dir = presets / preset_id / "templates"
template_dir.mkdir(parents=True)
(template_dir / f"{TEMPLATE}.md").write_text(
f"# {preset_id}\n",
encoding="utf-8",
)
registry_presets[preset_id] = metadata
(presets / ".registry").write_text(
json.dumps(registry, separators=(",", ":")) + "\n",
encoding="utf-8",
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TEMPLATE_CONTENT"] == expected
for result in results
)
@requires_bash
def test_all_variants_fail_when_wrap_placeholder_is_missing(
tmp_path: Path,
) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
templates = repo / ".specify" / "templates"
templates.mkdir(parents=True)
(templates / f"{TEMPLATE}.md").write_text("# Core\n", encoding="utf-8")
preset = repo / ".specify" / "presets" / "wrap-pack"
(preset / "templates").mkdir(parents=True)
(preset / "templates" / f"{TEMPLATE}.md").write_text(
"# Broken wrapper\n", encoding="utf-8"
)
(preset / "preset.yml").write_text(
"provides:\n"
" templates:\n"
" - type: template\n"
f" name: {TEMPLATE}\n"
f" file: templates/{TEMPLATE}.md\n"
" strategy: wrap\n",
encoding="utf-8",
)
(repo / ".specify" / "presets" / ".registry").write_text(
'{"presets":{"wrap-pack":{"enabled":true,"priority":1}}}\n',
encoding="utf-8",
)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode != 0 for result in results)
assert all(result.stdout == "" for result in results)
@requires_bash
def test_all_variants_fail_when_yaml_parser_is_unavailable(
tmp_path: Path,
) -> None:
repo, _ = _setup_repo(tmp_path)
blocker = tmp_path / "blocker"
blocker.mkdir()
(blocker / "yaml.py").write_text(
"raise ImportError('simulated missing PyYAML')\n",
encoding="utf-8",
)
env = clean_env()
env["PYTHONPATH"] = str(blocker)
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env),
]
if HAS_POWERSHELL:
results.append(
run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo, env)
)
assert all(result.returncode != 0 for result in results)
assert all(result.stdout == "" for result in results)
@requires_bash
def test_bash_fails_when_override_read_fails(tmp_path: Path) -> None:
repo = make_repo(tmp_path)
install_scripts(repo, SCRIPT)
override = repo / ".specify" / "templates" / "overrides"
override.mkdir(parents=True)
(override / f"{TEMPLATE}.md").write_text("# Override\n", encoding="utf-8")
shim_dir = tmp_path / "bin"
shim_dir.mkdir()
cat_shim = shim_dir / "cat"
cat_shim.write_text(
"#!/bin/sh\n"
"case \"$1\" in\n"
" */.specify/templates/overrides/*) exit 1 ;;\n"
"esac\n"
"exec /bin/cat \"$@\"\n",
encoding="utf-8",
)
cat_shim.chmod(0o755)
env = clean_env()
env["PATH"] = f"{shim_dir}{os.pathsep}{env.get('PATH', '')}"
result = run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo, env)
assert result.returncode != 0
assert result.stdout == ""
@requires_bash
@pytest.mark.parametrize(
"manifest_content",
[
"provides: [\n",
"",
"provides:\n templates:\n - null\n",
"provides:\n templates: {}\n",
"preset:\n id: wrap-pack\n",
"provides:\n templates: []\n",
f"""provides:
templates:
- type: template
name: {TEMPLATE}
file: null
strategy: wrap
""",
f"""provides:
templates:
- type: template
name: {TEMPLATE}
file: templates/{TEMPLATE}.md
strategy: 123
""",
f"""provides:
templates:
- type: template
name: {TEMPLATE}
file: templates/{TEMPLATE}.md
strategy: wrap
- type: template
name: unrelated-template
file: null
strategy: append
""",
f"""provides:
templates:
- name: {TEMPLATE}
file: templates/{TEMPLATE}.md
strategy: wrap
- type: template
name: unrelated-template
file: templates/other.md
""",
f"""provides:
templates:
- type: template
name: {TEMPLATE}
file: templates/{TEMPLATE}.md
strategy: wrap
- type: template
name: unrelated-template
""",
f"""provides:
templates:
- type: template
name: {TEMPLATE}
file: templates/{TEMPLATE}.md
strategy: wrap
- type: bogus
name: unrelated-template
file: templates/other.md
""",
f"""provides:
templates:
- type: template
name: {TEMPLATE}
file: templates/{TEMPLATE}.md
strategy: wrap
- type: template
name: unrelated-template
file: templates/other.md
strategy: merge
""",
],
ids=[
"invalid_yaml",
"empty_document",
"non_mapping_template_entry",
"non_list_templates",
"missing_provides",
"empty_templates",
"non_string_file",
"non_string_strategy",
"malformed_entry_after_match",
"entry_missing_type",
"entry_missing_file",
"unsupported_type",
"unsupported_strategy",
],
)
def test_all_variants_fail_for_malformed_preset_manifest(
tmp_path: Path,
manifest_content: str,
) -> None:
repo, _ = _setup_repo(tmp_path)
(
repo / ".specify" / "presets" / "wrap-pack" / "preset.yml"
).write_text(manifest_content, encoding="utf-8")
results = [
run(bash_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
run(py_cmd(repo, SCRIPT, TEMPLATE, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, TEMPLATE, "-Json"), repo))
assert all(result.returncode != 0 for result in results)
assert all(result.stdout == "" for result in results)
tests/test_setup_plan_python_parity.py
+69 -11
View File
@@ -11,7 +11,9 @@ from tests.parity_helpers import (
HAS_POWERSHELL,
POWERSHELL_EXE,
bash_cmd,
break_wrap_layer,
clean_env,
install_composition_stack,
install_scripts,
json_stdout,
make_repo,
@@ -60,7 +62,57 @@ def test_python_fresh_copy_matches_bash(tmp_path: Path) -> None:
)
for repo in (repo_a, repo_b):
plan = repo / "specs" / "001-my-feature" / "plan.md"
assert plan.read_text(encoding="utf-8") == TEMPLATE_BODY
assert plan.read_bytes() == TEMPLATE_BODY.encode("utf-8")
@requires_bash
def test_all_variants_materialize_composed_plan_template(tmp_path: Path) -> None:
repos = [
_setup_repo(tmp_path, "bash"),
_setup_repo(tmp_path, "powershell"),
_setup_repo(tmp_path, "python"),
]
expected = ""
for current in repos:
expected = install_composition_stack(
current, "plan-template", TEMPLATE_BODY
)
results = [
run(bash_cmd(repos[0], SCRIPT, "--json"), repos[0]),
run(py_cmd(repos[2], SCRIPT, "--json"), repos[2]),
]
checked_repos = [repos[0], repos[2]]
if HAS_POWERSHELL:
results.insert(1, run(ps_cmd(repos[1], SCRIPT, "-Json"), repos[1]))
checked_repos.insert(1, repos[1])
assert all(result.returncode == 0 for result in results)
for current in checked_repos:
assert (
current / "specs" / "001-my-feature" / "plan.md"
).read_text(encoding="utf-8") == expected
@requires_bash
def test_all_variants_fail_for_broken_plan_composition(tmp_path: Path) -> None:
repos = [
_setup_repo(tmp_path, "bash"),
_setup_repo(tmp_path, "powershell"),
_setup_repo(tmp_path, "python"),
]
for current in repos:
install_composition_stack(current, "plan-template", TEMPLATE_BODY)
break_wrap_layer(current, "plan-template")
results = [
run(bash_cmd(repos[0], SCRIPT, "--json"), repos[0]),
run(py_cmd(repos[2], SCRIPT, "--json"), repos[2]),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repos[1], SCRIPT, "-Json"), repos[1]))
assert all(result.returncode != 0 for result in results)
@requires_bash
@@ -119,13 +171,19 @@ def test_python_missing_template_matches_bash(tmp_path: Path) -> None:
@requires_bash
@pytest.mark.parametrize(
"registry",
("registry", "expected"),
[
'{"presets": {"alpha": {"priority": "high"}, "beta": {"priority": 1}}}',
'{"presets": {"alpha": {"priority": 2}, "beta": {"priority": 1}, "gamma": {"priority": null}}}',
"[]",
'{"presets":[]}',
'{"presets":null}',
(
'{"presets": {"alpha": {"priority": "high"}, "beta": {"priority": 1}}}',
"# beta plan\n",
),
(
'{"presets": {"alpha": {"priority": 2}, "beta": {"priority": 1}, "gamma": {"priority": null}}}',
"# beta plan\n",
),
("[]", "# alpha plan\n"),
('{"presets":[]}', "# alpha plan\n"),
('{"presets":null}', "# alpha plan\n"),
],
ids=[
"mixed_priorities",
@@ -135,10 +193,10 @@ def test_python_missing_template_matches_bash(tmp_path: Path) -> None:
"null_presets",
],
)
def test_all_variants_broken_registry_falls_back_to_dir_scan(
tmp_path: Path, registry: str
def test_all_variants_normalize_or_fallback_for_registry(
tmp_path: Path, registry: str, expected: str
) -> None:
"""Malformed registries fall back to the alphabetical directory scan."""
"""Priorities normalize canonically; malformed shapes fall back to directories."""
repos = [
_setup_repo(tmp_path, "bash", template=False),
_setup_repo(tmp_path, "powershell", template=False),
@@ -183,7 +241,7 @@ def test_all_variants_broken_registry_falls_back_to_dir_scan(
) == 1
for _, repo in results:
plan = repo / "specs" / "001-my-feature" / "plan.md"
assert plan.read_text(encoding="utf-8") == "# alpha plan\n"
assert plan.read_text(encoding="utf-8") == expected
@pytest.mark.skipif(not HAS_POWERSHELL, reason="no PowerShell available")
tests/test_setup_tasks.py
+1 -1
View File
@@ -719,7 +719,7 @@ def test_setup_tasks_ps_core_template_resolved(tasks_repo: Path) -> None:
[exe, "-NoProfile", "-File", str(script), "-Json"],
cwd=tasks_repo,
capture_output=True,
text=True,
encoding="utf-8",
check=False,
env=_clean_env(),
)
tests/test_setup_tasks_python_parity.py
+38
View File
@@ -10,7 +10,9 @@ from tests.conftest import requires_bash
from tests.parity_helpers import (
HAS_POWERSHELL,
bash_cmd,
break_wrap_layer,
clean_env,
install_composition_stack,
install_scripts,
json_stdout,
make_repo,
@@ -87,6 +89,42 @@ def test_python_override_template_wins_matches_bash(repo: Path) -> None:
assert json_stdout(py)["TASKS_TEMPLATE"].endswith("overrides/tasks-template.md")
@requires_bash
def test_all_variants_return_composed_tasks_template(repo: Path) -> None:
expected = install_composition_stack(
repo, "tasks-template", "# Tasks Template\n"
)
results = [
run(bash_cmd(repo, SCRIPT, "--json"), repo),
run(py_cmd(repo, SCRIPT, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, "-Json"), repo))
assert all(result.returncode == 0 for result in results)
assert all(
json_stdout(result)["TASKS_TEMPLATE_CONTENT"] == expected
for result in results
)
@requires_bash
def test_all_variants_fail_for_broken_tasks_composition(repo: Path) -> None:
install_composition_stack(repo, "tasks-template", "# Tasks Template\n")
break_wrap_layer(repo, "tasks-template")
results = [
run(bash_cmd(repo, SCRIPT, "--json"), repo),
run(py_cmd(repo, SCRIPT, "--json"), repo),
]
if HAS_POWERSHELL:
results.append(run(ps_cmd(repo, SCRIPT, "-Json"), repo))
assert all(result.returncode != 0 for result in results)
assert all(result.stdout == "" for result in results)
@requires_bash
@pytest.mark.parametrize(
"missing",