Git integration and Project workspace fixes

This commit is contained in:
Renn F
2026-01-08 16:45:25 +01:00
parent f1c5b7958c
commit 955455d704
28 changed files with 976 additions and 55 deletions
+27
View File
@@ -147,6 +147,29 @@ When a developer claims a git-enabled task, a **WorkSession** is created that tr
- PR number/URL when created - PR number/URL when created
- Merge status and who merged - Merge status and who merged
### Git Credentials
Git authentication is managed **per-project** through encrypted GitHub PATs:
- **Each project stores its own git token** - no global fallback
- **Tokens are encrypted at rest** using Fernet symmetric encryption
- **API never exposes tokens** - only returns `has_git_token: boolean`
- **Self-service via UI** - users set/update tokens in project settings
**Project fields:**
| Field | Description |
|-------|-------------|
| `git_token_encrypted` | Fernet-encrypted GitHub PAT (DB column) |
| `has_git_token` | Boolean indicator for API responses |
**Token flow:**
1. User creates project in UI, enters GitHub PAT
2. Token encrypted and stored in `projects.git_token_encrypted`
3. WorkspaceService decrypts token when cloning repos
4. GitService decrypts token for PR operations (gh CLI)
**HTTPS URLs require tokens** - attempting to clone without a token will raise `WorkspaceError`.
## Task Lifecycle ## Task Lifecycle
### Task States ### Task States
@@ -320,6 +343,10 @@ ROBOCO_DATABASE_NAME=roboco
ROBOCO_REDIS_HOST=localhost ROBOCO_REDIS_HOST=localhost
ROBOCO_REDIS_PORT=6379 ROBOCO_REDIS_PORT=6379
# Security (REQUIRED)
# Generate with: python -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())'
ROBOCO_ENCRYPTION_KEY=<your-fernet-key>
# Workspaces # Workspaces
ROBOCO_WORKSPACES_ROOT=/data/workspaces ROBOCO_WORKSPACES_ROOT=/data/workspaces
ROBOCO_WORKSPACE_AUTO_CLONE=true ROBOCO_WORKSPACE_AUTO_CLONE=true
+27 -2
View File
@@ -63,15 +63,34 @@ roboco_git_create_branch(project_slug, task_id, branch_type, parent_branch)
``` ```
- **branch_type**: `feature`, `bug`, `chore`, `docs`, `hotfix` - **branch_type**: `feature`, `bug`, `chore`, `docs`, `hotfix`
- **parent_branch**: Parent task's branch (or `main` for top-level) - **parent_branch**: Parent task's branch (or project's default branch for top-level)
- Branch naming: `{type}/{team}/{task_id}` (auto-generated) - Branch naming: `{type}/{team}/{task_id}` (auto-generated)
**Example:** **Example:**
``` ```
roboco_git_create_branch("roboco", "abc123", "feature", "main") roboco_git_create_branch("roboco", "abc123", "feature")
# Creates: feature/backend/abc123 # Creates: feature/backend/abc123
``` ```
**Branch Hierarchy (when using hierarchical branching):**
Branches are typically created top-down:
1. Main PM creates root branch from default branch
2. You create subtask branch from root branch
3. Dev works on subsubtask branch from your branch
**When creating a subtask branch:**
- Check if parent task has a branch set
- If so, use parent's branch as `parent_branch` parameter
- If branching from default branch directly, that works too
**Branch hierarchy example:**
```
default (main/master/etc)
└─ feature/backend/ROOT123 ← Main PM's branch
└─ feature/backend/ROOT123/SUB456 ← Your branch
└─ feature/backend/ROOT123/SUB456/DEV789 ← Dev branch
```
### 6. ACTIVATE ### 6. ACTIVATE
`roboco_task_activate()` moves backlog → pending. Now visible to devs. `roboco_task_activate()` moves backlog → pending. Now visible to devs.
@@ -175,6 +194,12 @@ When ALL subtasks done: reflect + complete your task.
6. **Pause after delegating** - Don't spin waiting 6. **Pause after delegating** - Don't spin waiting
7. **Reflect before complete** - `roboco_journal_reflect()` required 7. **Reflect before complete** - `roboco_journal_reflect()` required
**Task Delegation Options:**
- `roboco_task_assign()` - Reassign an existing task to a different agent
- `roboco_task_create(parent_task_id=...)` - Create a subtask under your task
For coordination tasks where you're managing work, subtasks are often cleaner for tracking.
## CEO Escalation ## CEO Escalation
For major tasks, escalate to CEO instead of completing directly: For major tasks, escalate to CEO instead of completing directly:
+12 -2
View File
@@ -30,13 +30,23 @@ Claim → read full description → plan breakdown across cells → start → jo
### 3. CREATE PARENT BRANCH (Git Tasks) ### 3. CREATE PARENT BRANCH (Git Tasks)
**For tasks with `requires_git=True`:** **For tasks with `requires_git=True`:**
``` ```
roboco_git_create_branch(project_slug, task_id, branch_type, "main") roboco_git_create_branch(project_slug, task_id, branch_type)
``` ```
- Creates parent branch from `main` - Creates parent branch from the project's default branch (e.g., `main`, `master`)
- Cell PM subtask branches will fork from this - Cell PM subtask branches will fork from this
- Example: `feature/cross/abc123` for cross-cell work - Example: `feature/cross/abc123` for cross-cell work
**Branch Hierarchy for Git Tasks:**
- For hierarchical branching: default branch → Your branch → Cell PM branch → Dev branch
- If using hierarchical branches, create root branch before Cell PMs create theirs
- Cell PM branches fork from your branch (set as `parent_branch`)
**Typical order for hierarchical branching:**
1. Create your root branch from project's default branch
2. Create cell tasks, Cell PMs create branches from your branch
3. Devs create branches from Cell PM branches
### 4. CREATE GROUP ### 4. CREATE GROUP
Use `roboco_group_create()` in each relevant cell channel. Cell PMs need groups to create sessions. Use `roboco_group_create()` in each relevant cell channel. Cell PMs need groups to create sessions.
+2
View File
@@ -105,6 +105,8 @@ def upgrade() -> None:
sa.Column( sa.Column(
"protected_branches", postgresql.ARRAY(sa.String()), server_default="{}" "protected_branches", postgresql.ARRAY(sa.String()), server_default="{}"
), ),
# Git Authentication (encrypted)
sa.Column("git_token_encrypted", sa.Text(), nullable=True),
# CI/CD Commands (optional) # CI/CD Commands (optional)
sa.Column("test_command", sa.String(500), nullable=True), sa.Column("test_command", sa.String(500), nullable=True),
sa.Column("lint_command", sa.String(500), nullable=True), sa.Column("lint_command", sa.String(500), nullable=True),
+260
View File
@@ -0,0 +1,260 @@
services:
# ==========================================================================
# PostgreSQL - Primary Database with pgvector for RAG
# ==========================================================================
postgres:
image: pgvector/pgvector:pg16
container_name: roboco-postgres
restart: unless-stopped
environment:
POSTGRES_USER: roboco
POSTGRES_PASSWORD: roboco
POSTGRES_DB: roboco
ports:
- "15432:5432"
volumes:
- ${ROBOCO_DATA_DIR:-./data}/postgres:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U roboco -d roboco"]
interval: 10s
timeout: 5s
retries: 5
# ==========================================================================
# Redis - Cache, Sessions, Event Bus
# ==========================================================================
redis:
image: redis:8-alpine
container_name: roboco-redis
restart: unless-stopped
command: redis-server --appendonly yes
ports:
- "16379:6379"
volumes:
- ${ROBOCO_DATA_DIR:-./data}/redis:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
# ==========================================================================
# Ollama - Local LLM and Embedding Server
# ==========================================================================
ollama:
image: ollama/ollama:latest
container_name: roboco-ollama
restart: unless-stopped
environment:
OLLAMA_API_KEY: ${OLLAMA_API_KEY}
ports:
- "11435:11434"
volumes:
- ${ROBOCO_DATA_DIR:-./data}/ollama:/root/.ollama
healthcheck:
# Use ollama CLI (guaranteed available) to check if server is responding
test: ["CMD", "ollama", "list"]
interval: 10s
timeout: 5s
retries: 5
start_period: 10s
# Ollama model puller - pulls required models on startup
# Uses streaming curl to wait for full model download
ollama-init:
image: curlimages/curl:latest
container_name: roboco-ollama-init
depends_on:
ollama:
condition: service_healthy
restart: "no"
entrypoint: ["/bin/sh", "-c"]
command:
- |
set -e
echo "=== Pulling embedding model (embeddinggemma:300m) ==="
# Ollama /api/pull streams JSON lines until complete - consume full stream
# Note: $$ escapes $ for docker-compose variable substitution
curl -sN http://ollama:11434/api/pull -d '{"name":"embeddinggemma:300m"}' | while read -r line; do
status=$$(echo "$$line" | grep -o '"status":"[^"]*"' | cut -d'"' -f4)
[ -n "$$status" ] && echo " $$status"
done
echo "=== Pulling LLM model (glm-4.7:cloud) ==="
curl -sN http://ollama:11434/api/pull -d '{"name":"glm-4.7:cloud"}' | while read -r line; do
status=$$(echo "$$line" | grep -o '"status":"[^"]*"' | cut -d'"' -f4)
[ -n "$$status" ] && echo " $$status"
done
echo "=== Verifying models are available ==="
curl -sf http://ollama:11434/api/tags | grep -q "embeddinggemma" && echo " embeddinggemma: OK"
curl -sf http://ollama:11434/api/tags | grep -q "glm-4.7" && echo " glm-4.7: OK"
echo "=== All models ready! ==="
# ==========================================================================
# Agent Base Image Builder (specialized images built on-demand by orchestrator)
# ==========================================================================
agent-base-image:
build:
context: .
dockerfile: docker/agent-base.Dockerfile
image: roboco-agent-base
container_name: roboco-agent-base-builder
entrypoint: ["/bin/sh", "-c", "echo 'Agent base image built successfully'"]
restart: "no"
# ==========================================================================
# Agent PM Image Builder (specialized image built on-demand by orchestrator)
# ==========================================================================
agent-pm-image:
build:
context: .
dockerfile: docker/agent-pm.Dockerfile
image: roboco-agent-pm
entrypoint: ["/bin/sh", "-c", "echo 'Agent PM image built'"]
restart: "no"
depends_on:
- agent-base-image
# ==========================================================================
# Agent Backend Dev Image Builder (specialized image built on-demand by orchestrator)
# ==========================================================================
agent-dev-be-image:
build:
context: .
dockerfile: docker/agent-dev-be.Dockerfile
image: roboco-agent-dev-be
entrypoint: ["/bin/sh", "-c", "echo 'Agent Backend Dev image built'"]
restart: "no"
depends_on:
- agent-base-image
# ==========================================================================
# Agent Frontend Dev Image Builder (specialized image built on-demand by orchestrator)
# ==========================================================================
agent-dev-fe-image:
build:
context: .
dockerfile: docker/agent-dev-fe.Dockerfile
image: roboco-agent-dev-fe
entrypoint: ["/bin/sh", "-c", "echo 'Agent Frontend Dev image built'"]
restart: "no"
depends_on:
- agent-base-image
# ==========================================================================
# Agent Backend QA Image Builder (specialized image built on-demand by orchestrator)
# ==========================================================================
agent-qa-be-image:
build:
context: .
dockerfile: docker/agent-qa-be.Dockerfile
image: roboco-agent-qa-be
entrypoint: ["/bin/sh", "-c", 'echo "Agent Backend QA image built"']
restart: "no"
depends_on:
- agent-base-image
# ==========================================================================
# Agent Frontend QA Image Builder (specialized image built on-demand by orchestrator)
# ==========================================================================
agent-qa-fe-image:
build:
context: .
dockerfile: docker/agent-qa-fe.Dockerfile
image: roboco-agent-qa-fe
entrypoint: ["/bin/sh", "-c", 'echo "Agent Frontend QA image built"']
restart: "no"
depends_on:
- agent-base-image
# ==========================================================================
# Agent UX/UI Dev Image Builder (specialized image built on-demand by orchestrator)
# ==========================================================================
agent-ux-image:
build:
context: .
dockerfile: docker/agent-ux.Dockerfile
image: roboco-agent-ux
entrypoint: ["/bin/sh", "-c", 'echo "Agent UX/UI image built"']
restart: "no"
depends_on:
- agent-base-image
# ==========================================================================
# Agent Documenter Image Builder (specialized image built on-demand by orchestrator)
# ==========================================================================
agent-doc-image:
build:
context: .
dockerfile: docker/agent-doc.Dockerfile
image: roboco-agent-doc
entrypoint: ["/bin/sh", "-c", 'echo "Agent Documenter image built"']
restart: "no"
depends_on:
- agent-base-image
# ==========================================================================
# Orchestrator - API Server + Agent Spawner
# ==========================================================================
orchestrator:
build:
context: .
dockerfile: docker/orchestrator.Dockerfile
image: roboco-orchestrator
container_name: roboco-orchestrator
restart: unless-stopped
ports:
- "8000:8000"
environment:
# Database (use container name, not localhost)
ROBOCO_DATABASE_HOST: roboco-postgres
ROBOCO_DATABASE_PORT: 5432
ROBOCO_DATABASE_USER: roboco
ROBOCO_DATABASE_PASSWORD: roboco
ROBOCO_DATABASE_NAME: roboco
# Redis (use container name)
ROBOCO_REDIS_HOST: roboco-redis
ROBOCO_REDIS_PORT: 6379
# API
ROBOCO_HOST: 0.0.0.0
ROBOCO_PORT: 8000
ROBOCO_ENCRYPTION_KEY: ${ROBOCO_ENCRYPTION_KEY:?ROBOCO_ENCRYPTION_KEY is required}
# Ollama (use container name)
ROBOCO_LOCAL_LLM_BASE_URL: http://roboco-ollama:11434/v1
ROBOCO_LOCAL_LLM_MODEL: glm-4.7:cloud
ROBOCO_DEFAULT_EMBEDDING_MODEL: embeddinggemma:300m
ROBOCO_OLLAMA_BASE_URL: http://roboco-ollama:11434
# Host paths for spawning agent containers (required for Docker-in-Docker)
# IMPORTANT: These must be ABSOLUTE paths on the host filesystem
ROBOCO_HOST_PROJECT_DIR: ${ROBOCO_HOST_PROJECT_DIR:-/volume1/roboco}
ROBOCO_HOST_CLAUDE_DIR: ${ROBOCO_HOST_CLAUDE_DIR:-/home/renzof/.claude}
ROBOCO_HOST_DATA_DIR: ${ROBOCO_HOST_DATA_DIR:-/volume1/roboco/data}
volumes:
# Docker socket - allows spawning agent containers
- /var/run/docker.sock:/var/run/docker.sock
# Claude Code auth - mount your ~/.claude directory
- ${CLAUDE_AUTH_DIR:-/home/renzof/.claude}:/root/.claude
# Shared config directory for MCP configs (writable)
- ${ROBOCO_DATA_DIR:-./data}/mcp-configs:/app/mcp-configs
# Generated prompts directory - composed at runtime from layers
- ${ROBOCO_DATA_DIR:-./data}/prompts-generated:/app/prompts-generated
# Per-agent Claude settings (generated at spawn time)
- ${ROBOCO_DATA_DIR:-./data}/agent-settings:/app/agent-settings
# Agent workspaces (git clones) - persisted across restarts
- ${ROBOCO_DATA_DIR:-./data}/workspaces:/data/workspaces
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
ollama:
condition: service_healthy
ollama-init:
condition: service_completed_successfully
agent-base-image:
condition: service_completed_successfully
# Default agents to spawn (override in .env or command line)
# command: ["--spawn", "main-pm", "be-dev-1", "be-qa"]
networks:
default:
name: roboco_default
+3
View File
@@ -217,6 +217,7 @@ services:
# API # API
ROBOCO_HOST: 0.0.0.0 ROBOCO_HOST: 0.0.0.0
ROBOCO_PORT: 8000 ROBOCO_PORT: 8000
ROBOCO_ENCRYPTION_KEY: ${ROBOCO_ENCRYPTION_KEY:?ROBOCO_ENCRYPTION_KEY is required}
# Ollama (use container name) # Ollama (use container name)
ROBOCO_LOCAL_LLM_BASE_URL: http://roboco-ollama:11434/v1 ROBOCO_LOCAL_LLM_BASE_URL: http://roboco-ollama:11434/v1
ROBOCO_LOCAL_LLM_MODEL: glm-4.7:cloud ROBOCO_LOCAL_LLM_MODEL: glm-4.7:cloud
@@ -238,6 +239,8 @@ services:
- ${ROBOCO_DATA_DIR:-./data}/prompts-generated:/app/prompts-generated - ${ROBOCO_DATA_DIR:-./data}/prompts-generated:/app/prompts-generated
# Per-agent Claude settings (generated at spawn time) # Per-agent Claude settings (generated at spawn time)
- ${ROBOCO_DATA_DIR:-./data}/agent-settings:/app/agent-settings - ${ROBOCO_DATA_DIR:-./data}/agent-settings:/app/agent-settings
# Agent workspaces (git clones) - persisted across restarts
- ${ROBOCO_DATA_DIR:-./data}/workspaces:/data/workspaces
depends_on: depends_on:
postgres: postgres:
condition: service_healthy condition: service_healthy
+11
View File
@@ -76,3 +76,14 @@ roboco_workspace_status(project_slug="roboco")
Path resolved automatically: `{workspaces_root}/{project}/{team}/{agent}/` Path resolved automatically: `{workspaces_root}/{project}/{team}/{agent}/`
If `auto_clone=True` and workspace doesn't exist, it's created on first access. If `auto_clone=True` and workspace doesn't exist, it's created on first access.
## Authentication
HTTPS repositories require a GitHub PAT configured on the project:
- **Token configured**: Auto-clone works, git operations succeed
- **Token missing**: Error "Project requires a git token for HTTPS repositories"
**If you see this error**: Contact your PM to configure the project's git token.
PMs use `roboco_project_update(slug, git_token="...")` to set credentials.
+10 -2
View File
@@ -113,11 +113,19 @@ Register new git repositories:
roboco_project_create( roboco_project_create(
name="New Project", name="New Project",
slug="new-project", slug="new-project",
git_url="git@github.com:org/repo.git", git_url="https://github.com/org/repo.git",
assigned_cell="backend" assigned_cell="backend",
git_token="ghp_xxxx..." # Required for HTTPS repos
) )
``` ```
**IMPORTANT:** Include `git_token` (GitHub PAT with `repo` scope) for HTTPS repositories. Without it, workspace creation and git operations will fail.
To update or rotate tokens:
```python
roboco_project_update(slug="new-project", git_token="ghp_newtoken...")
```
Create tasks with project: Create tasks with project:
```python ```python
+10 -2
View File
@@ -19,7 +19,9 @@ Returns projects you have access to (cell-scoped for non-PMs).
roboco_project_get(slug="roboco") roboco_project_get(slug="roboco")
``` ```
Returns: `name`, `git_url`, `assigned_cell`, `default_branch`, `test_command`, etc. Returns: `name`, `git_url`, `assigned_cell`, `default_branch`, `has_git_token`, `test_command`, etc.
**Note:** `has_git_token` indicates if authentication is configured (required for HTTPS repos).
## Create Project (PM+ Only) ## Create Project (PM+ Only)
@@ -27,8 +29,9 @@ Returns: `name`, `git_url`, `assigned_cell`, `default_branch`, `test_command`, e
roboco_project_create( roboco_project_create(
name="RoboCo Panel", name="RoboCo Panel",
slug="roboco-panel", slug="roboco-panel",
git_url="git@github.com:org/roboco-panel.git", git_url="https://github.com/org/roboco-panel.git",
assigned_cell="frontend", assigned_cell="frontend",
git_token="ghp_xxxx...", # GitHub PAT with repo scope
default_branch="main", default_branch="main",
test_command="pnpm test", test_command="pnpm test",
lint_command="pnpm lint" lint_command="pnpm lint"
@@ -37,11 +40,14 @@ roboco_project_create(
**Who can create:** Main PM, Board, CEO **Who can create:** Main PM, Board, CEO
**IMPORTANT:** `git_token` is **required** for HTTPS repositories. Without it, workspace creation and git operations will fail.
## Update Project ## Update Project
```python ```python
roboco_project_update( roboco_project_update(
slug="roboco-panel", slug="roboco-panel",
git_token="ghp_newtoken...", # Update/rotate token
test_command="pnpm test:ci", test_command="pnpm test:ci",
lint_command="pnpm lint:fix" lint_command="pnpm lint:fix"
) )
@@ -51,6 +57,8 @@ roboco_project_update(
- CEO, Main PM: Any project - CEO, Main PM: Any project
- Cell PM: Own cell's projects only - Cell PM: Own cell's projects only
**Token rotation:** Pass `git_token` to update credentials. Pass empty string to clear.
## Workspace Tools ## Workspace Tools
### Ensure Workspace ### Ensure Workspace
+16
View File
@@ -1,5 +1,21 @@
# Git Error Troubleshooting # Git Error Troubleshooting
## Missing Git Token
**Error**: "Project requires a git token for HTTPS repositories"
**Cause**: No GitHub PAT configured for this project
**Solutions**:
1. Open project settings in UI
2. Add GitHub token (Personal Access Token)
3. Token needs `repo` scope for clone/push/PR
**Notes**:
- Each project requires its own token (no global fallback)
- Tokens are encrypted at rest
- Token never exposed in API responses
## Workspace Not Found ## Workspace Not Found
**Error**: "Workspace does not exist" **Error**: "Workspace does not exist"
+6
View File
@@ -54,6 +54,12 @@ In `awaiting_documentation`:
Task advances to `awaiting_pm_review` when BOTH are done. Task advances to `awaiting_pm_review` when BOTH are done.
## Prerequisites
- **Git token configured**: Project must have a GitHub PAT set
- Token must have `repo` scope for PR creation
- If missing, error: "Project has no git token configured"
## Before Creating PR ## Before Creating PR
1. Push all commits: `roboco_git_push()` 1. Push all commits: `roboco_git_push()`
+23 -9
View File
@@ -96,9 +96,15 @@ async def get_git_status(
try: try:
workspace = await git_service.get_workspace(project_slug, agent.agent_id) workspace = await git_service.get_workspace(project_slug, agent.agent_id)
current_branch, has_changes, staged, unstaged, untracked, ahead, behind = ( (
await git_service.get_status(workspace) current_branch,
) has_changes,
staged,
unstaged,
untracked,
ahead,
behind,
) = await git_service.get_status(workspace)
except ServiceError as e: except ServiceError as e:
raise _translate_error(e) from e raise _translate_error(e) from e
@@ -273,9 +279,13 @@ async def create_commit(
try: try:
workspace = await git_service.get_workspace(data.project_slug, agent.agent_id) workspace = await git_service.get_workspace(data.project_slug, agent.agent_id)
commit_hash, message, files_changed, insertions, deletions = ( (
await git_service.create_commit(workspace, agent.agent_id, data) commit_hash,
) message,
files_changed,
insertions,
deletions,
) = await git_service.create_commit(workspace, agent.agent_id, data)
except ServiceError as e: except ServiceError as e:
raise _translate_error(e) from e raise _translate_error(e) from e
@@ -406,9 +416,13 @@ async def create_pull_request(
try: try:
workspace = await git_service.get_workspace(data.project_slug, agent.agent_id) workspace = await git_service.get_workspace(data.project_slug, agent.agent_id)
pr_number, pr_url, title, source_branch, target_branch = ( (
await git_service.create_pull_request(workspace, data) pr_number,
) pr_url,
title,
source_branch,
target_branch,
) = await git_service.create_pull_request(workspace, data)
except ServiceError as e: except ServiceError as e:
raise _translate_error(e) from e raise _translate_error(e) from e
+8 -1
View File
@@ -146,14 +146,20 @@ async def create_project(
service = get_project_service(db) service = get_project_service(db)
# If protected_branches wasn't provided, default to just the default_branch
protected_branches = data.protected_branches
if protected_branches is None:
protected_branches = [data.default_branch]
# Convert request to service model # Convert request to service model
create_data = ProjectCreate( create_data = ProjectCreate(
name=data.name, name=data.name,
slug=data.slug, slug=data.slug,
git_url=data.git_url, git_url=data.git_url,
default_branch=data.default_branch, default_branch=data.default_branch,
protected_branches=data.protected_branches, protected_branches=protected_branches,
assigned_cell=data.assigned_cell, assigned_cell=data.assigned_cell,
git_token=data.git_token,
test_command=data.test_command, test_command=data.test_command,
lint_command=data.lint_command, lint_command=data.lint_command,
format_command=data.format_command, format_command=data.format_command,
@@ -218,6 +224,7 @@ async def update_project(
default_branch=data.default_branch, default_branch=data.default_branch,
protected_branches=data.protected_branches, protected_branches=data.protected_branches,
assigned_cell=data.assigned_cell, assigned_cell=data.assigned_cell,
git_token=data.git_token,
test_command=data.test_command, test_command=data.test_command,
lint_command=data.lint_command, lint_command=data.lint_command,
format_command=data.format_command, format_command=data.format_command,
+17
View File
@@ -93,6 +93,19 @@ async def create_task(
detail="Not authorized to create tasks", detail="Not authorized to create tasks",
) )
# Validate: git tasks require project_id
if data.requires_git and not data.project_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail={
"error": {
"code": "PROJECT_REQUIRED",
"message": "Tasks with requires_git=True must have project_id set",
"hint": "Specify project_id or set requires_git=False",
}
},
)
service = get_task_service(db) service = get_task_service(db)
req = TaskCreateRequest( req = TaskCreateRequest(
title=data.title, title=data.title,
@@ -109,6 +122,10 @@ async def create_task(
status=data.status, status=data.status,
sequence=data.sequence, # Task ordering within siblings sequence=data.sequence, # Task ordering within siblings
dependency_ids=data.dependency_ids, # Dependencies for claim filtering dependency_ids=data.dependency_ids, # Dependencies for claim filtering
# Git configuration
task_type=data.task_type,
requires_git=data.requires_git,
project_id=data.project_id,
) )
task = await service.create(req) task = await service.create(req)
await db.commit() await db.commit()
+24 -1
View File
@@ -33,6 +33,9 @@ class ProjectResponse(BaseModel):
protected_branches: list[str] protected_branches: list[str]
assigned_cell: Team assigned_cell: Team
# Git authentication status (token never exposed, only boolean)
has_git_token: bool = False
# Optional commands # Optional commands
test_command: str | None = None test_command: str | None = None
lint_command: str | None = None lint_command: str | None = None
@@ -63,9 +66,11 @@ class ProjectSummaryResponse(BaseModel):
id: UUID id: UUID
name: str name: str
slug: str slug: str
git_url: str
assigned_cell: Team assigned_cell: Team
is_active: bool is_active: bool
has_workspace: bool = False has_workspace: bool = False
has_git_token: bool = False
class Config: class Config:
"""Pydantic config.""" """Pydantic config."""
@@ -85,9 +90,18 @@ class ProjectCreateRequest(BaseModel):
slug: str = Field(..., min_length=1, max_length=50, pattern=r"^[a-z0-9-]+$") slug: str = Field(..., min_length=1, max_length=50, pattern=r"^[a-z0-9-]+$")
git_url: str git_url: str
default_branch: str = "main" default_branch: str = "main"
protected_branches: list[str] = Field(default_factory=lambda: ["main", "master"]) protected_branches: list[str] | None = Field(
default=None,
description="Branches to protect. Defaults to [default_branch].",
)
assigned_cell: Team assigned_cell: Team
# Git authentication (will be encrypted and stored securely)
git_token: str | None = Field(
default=None,
description="GitHub PAT for clone/push/PR (stored encrypted, never returned)",
)
# Optional commands # Optional commands
test_command: str | None = None test_command: str | None = None
lint_command: str | None = None lint_command: str | None = None
@@ -105,6 +119,12 @@ class ProjectUpdateRequest(BaseModel):
protected_branches: list[str] | None = None protected_branches: list[str] | None = None
assigned_cell: Team | None = None assigned_cell: Team | None = None
# Git authentication (empty string clears token, None leaves unchanged)
git_token: str | None = Field(
default=None,
description="GitHub PAT (empty string clears, None leaves unchanged)",
)
# Commands # Commands
test_command: str | None = None test_command: str | None = None
lint_command: str | None = None lint_command: str | None = None
@@ -144,6 +164,7 @@ def project_to_response(project: "ProjectTable") -> ProjectResponse:
default_branch=str(default_branch) if default_branch else "main", default_branch=str(default_branch) if default_branch else "main",
protected_branches=list(project.protected_branches or []), protected_branches=list(project.protected_branches or []),
assigned_cell=project.assigned_cell, assigned_cell=project.assigned_cell,
has_git_token=bool(project.git_token_encrypted),
test_command=project.test_command, test_command=project.test_command,
lint_command=project.lint_command, lint_command=project.lint_command,
format_command=project.format_command, format_command=project.format_command,
@@ -165,7 +186,9 @@ def project_to_summary(project: "ProjectTable") -> ProjectSummaryResponse:
id=typing_cast("UUID", project.id), id=typing_cast("UUID", project.id),
name=str(project.name), name=str(project.name),
slug=str(project.slug), slug=str(project.slug),
git_url=str(project.git_url),
assigned_cell=project.assigned_cell, assigned_cell=project.assigned_cell,
is_active=bool(project.is_active), is_active=bool(project.is_active),
has_workspace=bool(project.workspace_path), has_workspace=bool(project.workspace_path),
has_git_token=bool(project.git_token_encrypted),
) )
+4
View File
@@ -198,6 +198,10 @@ class Settings(BaseSettings):
min_length=32, min_length=32,
description="Secret key for JWT signing", description="Secret key for JWT signing",
) )
encryption_key: str = Field(
default="",
description="Fernet encryption key for secrets.",
)
access_token_expire_minutes: int = Field(default=60 * 24, ge=1) # 24 hours access_token_expire_minutes: int = Field(default=60 * 24, ge=1) # 24 hours
algorithm: str = "HS256" algorithm: str = "HS256"
+3
View File
@@ -300,6 +300,9 @@ class ProjectTable(Base):
protected_branches: Mapped[list[str]] = mapped_column( protected_branches: Mapped[list[str]] = mapped_column(
ARRAY(String), default=lambda: ["main", "master"] ARRAY(String), default=lambda: ["main", "master"]
) )
git_token_encrypted: Mapped[str | None] = mapped_column(
Text, nullable=True
) # Fernet-encrypted GitHub PAT
# CI/CD Commands (optional) # CI/CD Commands (optional)
test_command: Mapped[str | None] = mapped_column(String(500), nullable=True) test_command: Mapped[str | None] = mapped_column(String(500), nullable=True)
+9 -2
View File
@@ -345,7 +345,10 @@ async def _handle_workspace_ensure(
"branch": git_status.get("current_branch"), "branch": git_status.get("current_branch"),
"has_uncommitted": git_status.get("has_changes", False), "has_uncommitted": git_status.get("has_changes", False),
}, },
"guidance": "Workspace ready. Use git tools to make changes.", "guidance": (
"Workspace ready. Use roboco_git_* MCP tools for git operations. "
f"Direct filesystem access at /data/workspaces/{project_slug}/..."
),
} }
@@ -384,7 +387,11 @@ async def _handle_workspace_status(
"staged_files": git_status.get("staged_files", []), "staged_files": git_status.get("staged_files", []),
"unstaged_files": git_status.get("unstaged_files", []), "unstaged_files": git_status.get("unstaged_files", []),
}, },
"guidance": "Use git tools to make changes.", "guidance": (
"Workspace ready. Use roboco_git_* MCP tools for git operations "
"(commit, push, branch, etc). Direct filesystem access is available "
f"at /data/workspaces/{project_slug}/... for your agent."
),
} }
+73 -8
View File
@@ -6,6 +6,8 @@ Handler for claiming tasks.
from typing import Any from typing import Any
from starlette import status
from roboco.agents_config import get_agent_role from roboco.agents_config import get_agent_role
from roboco.mcp.tasks import format_task_response from roboco.mcp.tasks import format_task_response
from roboco.mcp.tasks.handlers._helpers import ( from roboco.mcp.tasks.handlers._helpers import (
@@ -56,6 +58,51 @@ async def _is_pre_assigned_to_agent(
return agent_uuid is not None and str(assigned_to) == agent_uuid return agent_uuid is not None and str(assigned_to) == agent_uuid
async def _validate_branch_hierarchy(
client: ApiClient, task: dict[str, Any]
) -> dict[str, Any] | None:
"""Walk up task hierarchy ensuring all git-enabled ancestors have branches.
Returns error dict if hierarchy is incomplete, None if valid.
Branch hierarchy must be created from root down:
- Main PM creates root branch
- Cell PM creates subtask branch
- Developer works on subsubtask branch
"""
current_parent_id = task.get("parent_task_id")
ancestors_checked: list[str] = []
while current_parent_id:
parent_resp = await client.get(f"/tasks/{current_parent_id}")
if parent_resp.status_code != status.HTTP_200_OK:
# Can't fetch parent - may have been deleted, allow to proceed
break
parent = parent_resp.json()
parent_title = parent.get("title", str(current_parent_id)[:8])
ancestors_checked.append(parent_title)
# Check if parent is a git task missing its branch
if parent.get("requires_git") and not parent.get("branch_name"):
return format_error_response(
"BRANCH_HIERARCHY_INCOMPLETE",
f"Parent task '{parent_title}' has no branch yet.",
{
"missing_branch_task": str(current_parent_id),
"ancestors_checked": ancestors_checked,
},
hint=(
"Branch hierarchy must be created from root down. "
"Main PM creates root branch, Cell PM creates subtask branch. "
"Ask your PM to create the parent branch first."
),
)
current_parent_id = parent.get("parent_task_id")
return None
async def _execute_claim( async def _execute_claim(
client: ApiClient, task_id: str, agent_id: str client: ApiClient, task_id: str, agent_id: str
) -> tuple[dict[str, Any] | None, dict[str, Any] | None]: ) -> tuple[dict[str, Any] | None, dict[str, Any] | None]:
@@ -72,6 +119,29 @@ async def _execute_claim(
return claimed, None return claimed, None
async def _validate_git_requirements(
client: ApiClient, task: dict[str, Any], task_id: str
) -> dict[str, Any] | None:
"""Validate git-related requirements. Returns error or None."""
if not task.get("requires_git"):
return None
# Branch must exist for git tasks
if not task.get("branch_name"):
return format_error_response(
"BRANCH_REQUIRED",
"Cannot claim git task - PM must create branch first.",
{"task_id": task_id, "requires_git": True},
hint="PM should call roboco_git_create_branch() before developer can claim",
)
# Validate full branch hierarchy for git tasks with parent
if task.get("parent_task_id"):
return await _validate_branch_hierarchy(client, task)
return None
async def handle_task_claim( async def handle_task_claim(
client: ApiClient, task_id: str, agent_id: str client: ApiClient, task_id: str, agent_id: str
) -> dict[str, Any]: ) -> dict[str, Any]:
@@ -104,14 +174,9 @@ async def handle_task_claim(
if error := await validate_task_claimable(task, agent_role, agent_id, client): if error := await validate_task_claimable(task, agent_role, agent_id, client):
return error return error
# Validate branch exists for git tasks (PM must create branch first) # Validate git requirements (branch exists, hierarchy valid)
if task.get("requires_git") and not task.get("branch_name"): if error := await _validate_git_requirements(client, task, task_id):
return format_error_response( return error
"BRANCH_REQUIRED",
"Cannot claim git task - PM must create branch first.",
{"task_id": task_id, "requires_git": True},
hint="PM should call roboco_git_create_branch() before developer can claim",
)
# Execute the claim # Execute the claim
claimed_task, error = await _execute_claim(client, task_id, agent_id) claimed_task, error = await _execute_claim(client, task_id, agent_id)
+12
View File
@@ -451,6 +451,15 @@ async def handle_task_assign(
if assign_error or assigned_task is None: if assign_error or assigned_task is None:
return assign_error or format_error_response("ASSIGN_FAILED", "No task") return assign_error or format_error_response("ASSIGN_FAILED", "No task")
# Warning if reassigning a claimed task (not an error, just informational)
reassign_warning = ""
if task.get("status") == "claimed" and task.get("claimed_by"):
reassign_warning = (
"\n\n⚠️ Note: This task was already claimed. If you're delegating work, "
"consider using roboco_task_create(parent_task_id=...) to create a subtask "
"for better tracking and branch hierarchy management."
)
guidance = ( guidance = (
f"Task assigned to {input_data.assignee} and set to pending. " f"Task assigned to {input_data.assignee} and set to pending. "
"Orchestrator will spawn them to claim and work on it." "Orchestrator will spawn them to claim and work on it."
@@ -461,6 +470,9 @@ async def handle_task_assign(
if role_warning: if role_warning:
guidance += f"\n\n⚠️ ROLE WARNING: {role_warning}" guidance += f"\n\n⚠️ ROLE WARNING: {role_warning}"
# Add reassign warning if applicable
guidance += reassign_warning
return format_task_response(assigned_task, "ASSIGNED", guidance) return format_task_response(assigned_task, "ASSIGNED", guidance)
+6 -2
View File
@@ -160,10 +160,14 @@ async def handle_task_start(
# Build guidance based on whether git checkout happened # Build guidance based on whether git checkout happened
guidance = "Task started. Work through your plan step by step:\n" guidance = "Task started. Work through your plan step by step:\n"
if requires_git and branch_name: if requires_git and branch_name:
guidance += f"✓ Checked out branch: {branch_name}\n\n" guidance += (
f"✓ Checked out branch: {branch_name}\n"
f" Workspace: /data/workspaces/{project_slug}/...\n"
" Use roboco_git_* tools for git operations.\n\n"
)
guidance += ( guidance += (
"1. Implement each sub-task\n" "1. Implement each sub-task\n"
"2. Commit frequently with clear messages\n" "2. Use roboco_git_commit() to commit frequently\n"
"3. Call roboco_task_progress to update status\n" "3. Call roboco_task_progress to update status\n"
"4. If blocked, call roboco_task_block immediately\n" "4. If blocked, call roboco_task_block immediately\n"
"5. When done, call roboco_task_submit_verification" "5. When done, call roboco_task_submit_verification"
+19
View File
@@ -77,6 +77,12 @@ class Project(TimestampMixin):
default=None, description="Specific agents allowed (None = all in cell)" default=None, description="Specific agents allowed (None = all in cell)"
) )
# Git Authentication (token stored encrypted, never exposed)
has_git_token: bool = Field(
default=False,
description="Whether a git token is configured (token never exposed)",
)
# Runtime State # Runtime State
workspace_path: str | None = Field( workspace_path: str | None = Field(
default=None, default=None,
@@ -102,6 +108,12 @@ class ProjectCreate(RobocoBase):
protected_branches: list[str] = Field(default_factory=lambda: ["main", "master"]) protected_branches: list[str] = Field(default_factory=lambda: ["main", "master"])
assigned_cell: Team assigned_cell: Team
# Git authentication (will be encrypted and stored securely)
git_token: str | None = Field(
default=None,
description="GitHub PAT for clone/push/PR operations (stored encrypted)",
)
# Optional commands # Optional commands
test_command: str | None = None test_command: str | None = None
lint_command: str | None = None lint_command: str | None = None
@@ -117,6 +129,13 @@ class ProjectUpdate(RobocoBase):
git_url: str | None = None git_url: str | None = None
default_branch: str | None = None default_branch: str | None = None
protected_branches: list[str] | None = None protected_branches: list[str] | None = None
# Git authentication (empty string clears token, None leaves unchanged)
git_token: str | None = Field(
default=None,
description="GitHub PAT (empty string clears, None leaves unchanged)",
)
test_command: str | None = None test_command: str | None = None
lint_command: str | None = None lint_command: str | None = None
format_command: str | None = None format_command: str | None = None
+5
View File
@@ -655,6 +655,7 @@ class AgentOrchestrator:
# Running inside orchestrator container - use host paths # Running inside orchestrator container - use host paths
blueprints_host = f"{PROJECT_HOST_PATH}/agents/blueprints" blueprints_host = f"{PROJECT_HOST_PATH}/agents/blueprints"
docs_host = f"{PROJECT_HOST_PATH}/docs" docs_host = f"{PROJECT_HOST_PATH}/docs"
workspaces_host = f"{DATA_HOST_PATH}/workspaces"
claude_host = CLAUDE_AUTH_HOST_PATH claude_host = CLAUDE_AUTH_HOST_PATH
mcp_config_host = ( mcp_config_host = (
f"{DATA_HOST_PATH}/mcp-configs/{config.mcp_config_path.name}" f"{DATA_HOST_PATH}/mcp-configs/{config.mcp_config_path.name}"
@@ -674,6 +675,7 @@ class AgentOrchestrator:
# Running directly on host # Running directly on host
blueprints_host = str(self.blueprints_dir.absolute()) blueprints_host = str(self.blueprints_dir.absolute())
docs_host = str(self.blueprints_dir.parent / "docs") docs_host = str(self.blueprints_dir.parent / "docs")
workspaces_host = str(Path(settings.workspaces_root))
claude_host = CLAUDE_AUTH_HOST_PATH claude_host = CLAUDE_AUTH_HOST_PATH
mcp_config_host = str(config.mcp_config_path) mcp_config_host = str(config.mcp_config_path)
# Generated prompts in temp dir # Generated prompts in temp dir
@@ -721,6 +723,9 @@ class AgentOrchestrator:
# - All other roles get read-only access # - All other roles get read-only access
"-v", "-v",
f"{docs_host}:/app/docs{'' if config.agent_id in ALL_DOCS else ':ro'}", f"{docs_host}:/app/docs{'' if config.agent_id in ALL_DOCS else ':ro'}",
# Mount workspaces directory for git operations
"-v",
f"{workspaces_host}:/data/workspaces",
# Mount MCP config # Mount MCP config
"-v", "-v",
f"{mcp_config_host}:/app/mcp-config.json:ro", f"{mcp_config_host}:/app/mcp-config.json:ro",
+61 -6
View File
@@ -6,6 +6,7 @@ All business logic for git commands, commit templates, PR generation.
""" """
import asyncio import asyncio
import os
import subprocess import subprocess
from pathlib import Path from pathlib import Path
from typing import ClassVar from typing import ClassVar
@@ -49,6 +50,24 @@ from roboco.templates.git.pr_root import CommitInfo as PRCommitInfo
# Git command timeout in seconds # Git command timeout in seconds
_GIT_TIMEOUT = 30 _GIT_TIMEOUT = 30
def _get_gh_env(token: str | None = None) -> dict[str, str]:
"""
Get environment variables for gh CLI commands.
Args:
token: Project-specific GitHub PAT (required for gh operations)
Returns:
Environment dict with GITHUB_TOKEN set
"""
env = os.environ.copy()
if token:
env["GITHUB_TOKEN"] = token
env["GH_TOKEN"] = token
return env
# Expected number of parts in various git outputs # Expected number of parts in various git outputs
_REV_LIST_PARTS = 2 _REV_LIST_PARTS = 2
@@ -285,9 +304,7 @@ class GitService(BaseService):
commit_hash = parts[0] if parts else "unknown" commit_hash = parts[0] if parts else "unknown"
# Get stats # Get stats
stat_result = await self._run_git( stat_result = await self._run_git(workspace, ["diff", "--stat", "HEAD~1..HEAD"])
workspace, ["diff", "--stat", "HEAD~1..HEAD"]
)
insertions, deletions, files_changed = self._parse_commit_stats( insertions, deletions, files_changed = self._parse_commit_stats(
stat_result.stdout stat_result.stdout
) )
@@ -335,6 +352,25 @@ class GitService(BaseService):
project = await project_service.get_by_slug(request.project_slug) project = await project_service.get_by_slug(request.project_slug)
base_branch = str(project.default_branch) if project else "main" base_branch = str(project.default_branch) if project else "main"
# Validate parent branch exists on remote (unless it's the default branch)
project_service = get_project_service(self.session)
project = await project_service.get_by_slug(request.project_slug)
default_branch = str(project.default_branch) if project else "main"
if base_branch != default_branch:
# Parent is not the default branch - verify it exists on remote
result = await self._run_git(
workspace,
["ls-remote", "--heads", "origin", base_branch],
check=False,
)
if not result.stdout.strip():
raise ValidationError(
f"Parent branch '{base_branch}' does not exist on remote. "
f"The parent task's branch must be created and pushed first. "
f"Ensure Main PM/Cell PM created their branches before this task."
)
# Create and push branch # Create and push branch
await self._run_git(workspace, ["checkout", base_branch]) await self._run_git(workspace, ["checkout", base_branch])
await self._run_git(workspace, ["pull", "origin", base_branch]) await self._run_git(workspace, ["pull", "origin", base_branch])
@@ -498,11 +534,21 @@ class GitService(BaseService):
source_branch = await self.get_current_branch(workspace) source_branch = await self.get_current_branch(workspace)
# Determine target branch # Determine target branch and get project token
project_service = get_project_service(self.session) project_service = get_project_service(self.session)
project = await project_service.get_by_slug(request.project_slug) project = await project_service.get_by_slug(request.project_slug)
default_branch = str(project.default_branch) if project else "main" default_branch = str(project.default_branch) if project else "main"
# Get decrypted token from project (required for PR creation)
git_token = await project_service.get_decrypted_token_by_slug(
request.project_slug
)
if not git_token:
raise GitError(
f"Project '{request.project_slug}' has no git token configured. "
"Configure a GitHub PAT in the project settings to create PRs."
)
if request.is_root_pr: if request.is_root_pr:
target_branch = default_branch target_branch = default_branch
elif task.parent_task_id: elif task.parent_task_id:
@@ -531,7 +577,7 @@ class GitService(BaseService):
pr_title = pr_title or build_pr_title_internal(internal_ctx) pr_title = pr_title or build_pr_title_internal(internal_ctx)
pr_body = pr_body or build_pr_body_internal(internal_ctx, api_base) pr_body = pr_body or build_pr_body_internal(internal_ctx, api_base)
# Create PR using gh CLI # Create PR using gh CLI with project token
def _create_pr() -> subprocess.CompletedProcess[str]: def _create_pr() -> subprocess.CompletedProcess[str]:
return subprocess.run( return subprocess.run(
[ [
@@ -552,6 +598,7 @@ class GitService(BaseService):
text=True, text=True,
timeout=_GIT_TIMEOUT, timeout=_GIT_TIMEOUT,
check=True, check=True,
env=_get_gh_env(git_token),
) )
try: try:
@@ -575,6 +622,14 @@ class GitService(BaseService):
Returns: (target_branch, merge_commit) Returns: (target_branch, merge_commit)
""" """
# Get project token for gh CLI
project_service = get_project_service(self.session)
git_token = await project_service.get_decrypted_token_by_slug(project_slug)
if not git_token:
raise GitError(
f"Project '{project_slug}' has no git token configured. "
"Configure a GitHub PAT in the project settings to merge PRs."
)
def _merge_pr() -> subprocess.CompletedProcess[str]: def _merge_pr() -> subprocess.CompletedProcess[str]:
return subprocess.run( return subprocess.run(
@@ -591,6 +646,7 @@ class GitService(BaseService):
text=True, text=True,
timeout=_GIT_TIMEOUT, timeout=_GIT_TIMEOUT,
check=True, check=True,
env=_get_gh_env(git_token),
) )
try: try:
@@ -599,7 +655,6 @@ class GitService(BaseService):
raise GitCommandError("gh pr merge", e.stderr or e.stdout or "") from e raise GitCommandError("gh pr merge", e.stderr or e.stdout or "") from e
# Get target branch # Get target branch
project_service = get_project_service(self.session)
project = await project_service.get_by_slug(project_slug) project = await project_service.get_by_slug(project_slug)
target_branch = str(project.default_branch) if project else "main" target_branch = str(project.default_branch) if project else "main"
+102 -3
View File
@@ -15,6 +15,7 @@ from roboco.db.tables import ProjectTable
from roboco.models.base import Team from roboco.models.base import Team
from roboco.models.project import ProjectCreate, ProjectUpdate from roboco.models.project import ProjectCreate, ProjectUpdate
from roboco.services.base import BaseService, ConflictError, NotFoundError from roboco.services.base import BaseService, ConflictError, NotFoundError
from roboco.utils.crypto import EncryptionError, decrypt_token, encrypt_token
class ProjectService(BaseService): class ProjectService(BaseService):
@@ -60,6 +61,15 @@ class ProjectService(BaseService):
resource_type="project", resource_type="project",
) )
# Encrypt git token if provided
encrypted_token = None
if data.git_token:
try:
encrypted_token = encrypt_token(data.git_token)
except EncryptionError as e:
self.log.error("Failed to encrypt git token", error=str(e))
raise
project = ProjectTable( project = ProjectTable(
name=data.name, name=data.name,
slug=data.slug, slug=data.slug,
@@ -67,6 +77,7 @@ class ProjectService(BaseService):
default_branch=data.default_branch, default_branch=data.default_branch,
protected_branches=data.protected_branches, protected_branches=data.protected_branches,
assigned_cell=data.assigned_cell, assigned_cell=data.assigned_cell,
git_token_encrypted=encrypted_token,
test_command=data.test_command, test_command=data.test_command,
lint_command=data.lint_command, lint_command=data.lint_command,
format_command=data.format_command, format_command=data.format_command,
@@ -83,6 +94,7 @@ class ProjectService(BaseService):
project_id=str(project.id), project_id=str(project.id),
slug=data.slug, slug=data.slug,
git_url=data.git_url, git_url=data.git_url,
has_git_token=bool(encrypted_token),
cell=data.assigned_cell.value cell=data.assigned_cell.value
if isinstance(data.assigned_cell, Team) if isinstance(data.assigned_cell, Team)
else data.assigned_cell, else data.assigned_cell,
@@ -129,18 +141,42 @@ class ProjectService(BaseService):
if not project: if not project:
return None return None
# Apply updates for non-None fields # Handle git_token specially (empty string clears, None leaves unchanged)
update_data = data.model_dump(exclude_unset=True, exclude_none=True) token_updated = False
if data.git_token is not None:
if data.git_token == "":
# Clear the token
project.git_token_encrypted = None
token_updated = True
self.log.info("Git token cleared", project_id=str(project_id))
else:
# Encrypt and set new token
try:
project.git_token_encrypted = encrypt_token(data.git_token)
token_updated = True
self.log.info("Git token updated", project_id=str(project_id))
except EncryptionError as e:
self.log.error("Failed to encrypt git token", error=str(e))
raise
# Apply updates for non-None fields (excluding git_token which we handled)
update_data = data.model_dump(
exclude_unset=True, exclude_none=True, exclude={"git_token"}
)
for key, value in update_data.items(): for key, value in update_data.items():
if hasattr(project, key): if hasattr(project, key):
setattr(project, key, value) setattr(project, key, value)
await self.session.flush() await self.session.flush()
updated_fields = list(update_data.keys())
if token_updated:
updated_fields.append("git_token")
self.log.info( self.log.info(
"Project updated", "Project updated",
project_id=str(project_id), project_id=str(project_id),
updates=list(update_data.keys()), updates=updated_fields,
) )
return project return project
@@ -287,6 +323,69 @@ class ProjectService(BaseService):
) )
return project return project
# =========================================================================
# GIT TOKEN MANAGEMENT
# =========================================================================
async def get_decrypted_token(self, project_id: UUID) -> str | None:
"""
Get the decrypted git token for a project.
Used by WorkspaceService and GitService for git operations.
The token is decrypted on-demand and should not be cached.
Args:
project_id: Project to get token for
Returns:
Decrypted token or None if no token is set
Raises:
EncryptionError: If decryption fails (key mismatch, corrupted data)
"""
project = await self.get(project_id)
if not project or not project.git_token_encrypted:
return None
try:
return decrypt_token(project.git_token_encrypted)
except EncryptionError:
self.log.error(
"Failed to decrypt git token",
project_id=str(project_id),
error="encryption_key_mismatch_or_corrupted",
)
raise
async def get_decrypted_token_by_slug(self, slug: str) -> str | None:
"""
Get the decrypted git token for a project by slug.
Convenience method for services that work with project slugs.
Args:
slug: Project slug
Returns:
Decrypted token or None if no token is set
Raises:
EncryptionError: If decryption fails
"""
project = await self.get_by_slug(slug)
if not project or not project.git_token_encrypted:
return None
try:
return decrypt_token(project.git_token_encrypted)
except EncryptionError:
self.log.error(
"Failed to decrypt git token",
project_slug=slug,
error="encryption_key_mismatch_or_corrupted",
)
raise
# ========================================================================= # =========================================================================
# ACCESS CONTROL # ACCESS CONTROL
# ========================================================================= # =========================================================================
+106 -15
View File
@@ -19,6 +19,7 @@ Example:
""" """
import asyncio import asyncio
import re
import subprocess import subprocess
from pathlib import Path from pathlib import Path
from uuid import UUID from uuid import UUID
@@ -26,13 +27,42 @@ from uuid import UUID
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from roboco.config import settings from roboco.config import settings
from roboco.db.tables import AgentTable, ProjectTable from roboco.db.tables import AgentTable
from roboco.logging import get_logger from roboco.logging import get_logger
from roboco.models.base import Team from roboco.models.base import Team
logger = get_logger(__name__) logger = get_logger(__name__)
def _inject_token_into_url(git_url: str, token: str | None) -> str:
"""
Inject GitHub PAT into HTTPS git URL for authentication.
Args:
git_url: Original git URL (SSH or HTTPS)
token: GitHub PAT (if None, returns original URL)
Returns:
URL with embedded token for HTTPS, or original URL for SSH
Example:
https://github.com/org/repo.git -> https://TOKEN@github.com/org/repo.git
"""
if not token:
return git_url
# Only inject for HTTPS URLs
if not git_url.startswith("https://"):
return git_url
# Check if token already present
if "@" in git_url.split("//")[1].split("/")[0]:
return git_url
# Inject token: https://github.com -> https://TOKEN@github.com
return re.sub(r"^https://", f"https://{token}@", git_url)
class WorkspaceError(Exception): class WorkspaceError(Exception):
"""Raised when workspace operations fail.""" """Raised when workspace operations fail."""
@@ -145,7 +175,27 @@ class WorkspaceService:
Raises: Raises:
WorkspaceError: If workspace creation fails WorkspaceError: If workspace creation fails
""" """
workspace = await self.resolve_workspace(project_slug, agent_id) from sqlalchemy import select
from roboco.services.project import get_project_service
# Look up agent for workspace path and git identity
agent_id_str = str(agent_id)
query = select(AgentTable)
try:
agent_uuid = UUID(agent_id_str)
query = query.where(AgentTable.id == agent_uuid)
except ValueError:
query = query.where(AgentTable.slug == agent_id_str)
result = await self.session.execute(query)
agent = result.scalar_one_or_none()
if not agent:
raise WorkspaceError(f"Agent not found: {agent_id}")
# Compute workspace path
team = agent.team if agent.team else Team.BACKEND
workspace = self.get_workspace_path(project_slug, team, agent.slug)
# Check if already exists # Check if already exists
if (workspace / ".git").exists(): if (workspace / ".git").exists():
@@ -156,21 +206,34 @@ class WorkspaceService:
) )
return workspace return workspace
# Get git URL if not provided # Get git URL and token from project
if not git_url: project_service = get_project_service(self.session)
from sqlalchemy import select project = await project_service.get_by_slug(project_slug)
if not project:
raise WorkspaceError(f"Project not found: {project_slug}")
result = await self.session.execute( if not git_url:
select(ProjectTable).where(ProjectTable.slug == project_slug)
)
project = result.scalar_one_or_none()
if not project:
raise WorkspaceError(f"Project not found: {project_slug}")
git_url = project.git_url git_url = project.git_url
default_branch = project.default_branch or default_branch default_branch = project.default_branch or default_branch
# Clone the repository # Get decrypted token from project (per-project token, no global fallback)
await self._clone_repo(workspace, git_url, default_branch) git_token = await project_service.get_decrypted_token_by_slug(project_slug)
# Validate token is set for HTTPS URLs (no global fallback)
if git_url.startswith("https://") and not git_token:
raise WorkspaceError(
f"Project '{project_slug}' requires a git token for HTTPS clone. "
"Configure a GitHub PAT in the project settings."
)
# Clone the repository with agent identity
await self._clone_repo(
workspace,
git_url,
default_branch,
git_token,
agent=agent,
)
return workspace return workspace
async def _clone_repo( async def _clone_repo(
@@ -178,6 +241,8 @@ class WorkspaceService:
workspace: Path, workspace: Path,
git_url: str, git_url: str,
default_branch: str, default_branch: str,
git_token: str | None = None,
agent: AgentTable | None = None,
) -> None: ) -> None:
""" """
Clone a git repository to the workspace. Clone a git repository to the workspace.
@@ -186,6 +251,8 @@ class WorkspaceService:
workspace: Target directory workspace: Target directory
git_url: Git URL to clone git_url: Git URL to clone
default_branch: Branch to checkout default_branch: Branch to checkout
git_token: GitHub PAT for authentication (per-project)
agent: Agent for git identity (name/email in commits)
Raises: Raises:
WorkspaceError: If clone fails WorkspaceError: If clone fails
@@ -193,11 +260,16 @@ class WorkspaceService:
# Create parent directories # Create parent directories
workspace.parent.mkdir(parents=True, exist_ok=True) workspace.parent.mkdir(parents=True, exist_ok=True)
# Inject project-specific token for HTTPS URLs
auth_url = _inject_token_into_url(git_url, git_token)
# Log without exposing token
logger.info( logger.info(
"Cloning repository", "Cloning repository",
workspace=str(workspace), workspace=str(workspace),
git_url=git_url, git_url=git_url, # Log original URL, not auth URL
branch=default_branch, branch=default_branch,
using_token=bool(git_token and auth_url != git_url),
) )
def _do_clone() -> subprocess.CompletedProcess[str]: def _do_clone() -> subprocess.CompletedProcess[str]:
@@ -208,7 +280,7 @@ class WorkspaceService:
"--branch", "--branch",
default_branch, default_branch,
"--single-branch", "--single-branch",
git_url, auth_url,
str(workspace), str(workspace),
], ],
capture_output=True, capture_output=True,
@@ -217,8 +289,27 @@ class WorkspaceService:
check=True, check=True,
) )
def _configure_git() -> None:
"""Configure git author info based on agent identity."""
name = agent.name if agent else "RoboCo Agent"
slug = agent.slug if agent else "agent"
subprocess.run(
["git", "config", "user.name", name],
cwd=str(workspace),
check=True,
capture_output=True,
)
subprocess.run(
["git", "config", "user.email", f"{slug}@agents.roboco.dev"],
cwd=str(workspace),
check=True,
capture_output=True,
)
try: try:
await asyncio.to_thread(_do_clone) await asyncio.to_thread(_do_clone)
await asyncio.to_thread(_configure_git)
logger.info( logger.info(
"Repository cloned successfully", "Repository cloned successfully",
workspace=str(workspace), workspace=str(workspace),
+10
View File
@@ -5,8 +5,18 @@ Common utility functions and helpers.
""" """
from roboco.utils.converters import require_uuid, to_python_uuid, to_python_uuid_list from roboco.utils.converters import require_uuid, to_python_uuid, to_python_uuid_list
from roboco.utils.crypto import (
EncryptionError,
decrypt_token,
encrypt_token,
is_encryption_configured,
)
__all__ = [ __all__ = [
"EncryptionError",
"decrypt_token",
"encrypt_token",
"is_encryption_configured",
"require_uuid", "require_uuid",
"to_python_uuid", "to_python_uuid",
"to_python_uuid_list", "to_python_uuid_list",
+110
View File
@@ -0,0 +1,110 @@
"""
Cryptographic utilities for encrypting sensitive data at rest.
Uses Fernet symmetric encryption with a master key from settings.
"""
from cryptography.fernet import Fernet, InvalidToken
from roboco.config import settings
from roboco.logging import get_logger
logger = get_logger(__name__)
class EncryptionError(Exception):
"""Raised when encryption/decryption operations fail."""
pass
def _get_fernet() -> Fernet:
"""
Get Fernet instance with master encryption key.
Raises:
EncryptionError: If encryption key is not configured
"""
if not settings.encryption_key:
raise EncryptionError(
"ROBOCO_ENCRYPTION_KEY is not configured. "
"Generate one with: python -c 'from cryptography.fernet "
"import Fernet; print(Fernet.generate_key().decode())'"
)
try:
return Fernet(settings.encryption_key.encode())
except Exception as e:
raise EncryptionError(f"Invalid encryption key format: {e}") from e
def encrypt_token(token: str) -> str:
"""
Encrypt a token using Fernet symmetric encryption.
Args:
token: The plaintext token to encrypt
Returns:
Base64-encoded encrypted token string
Raises:
EncryptionError: If encryption fails
"""
if not token:
raise EncryptionError("Cannot encrypt empty token")
try:
fernet = _get_fernet()
encrypted = fernet.encrypt(token.encode())
return encrypted.decode()
except EncryptionError:
raise
except Exception as e:
logger.error("Token encryption failed", error=str(e))
raise EncryptionError(f"Failed to encrypt token: {e}") from e
def decrypt_token(encrypted: str) -> str:
"""
Decrypt a token that was encrypted with encrypt_token().
Args:
encrypted: Base64-encoded encrypted token string
Returns:
The original plaintext token
Raises:
EncryptionError: If decryption fails (wrong key, corrupted data, etc.)
"""
if not encrypted:
raise EncryptionError("Cannot decrypt empty value")
try:
fernet = _get_fernet()
decrypted = fernet.decrypt(encrypted.encode())
return decrypted.decode()
except InvalidToken as e:
logger.error(
"Token decryption failed - encryption key may have changed",
error="InvalidToken",
)
raise EncryptionError(
"Unable to decrypt token - encryption key may have changed"
) from e
except EncryptionError:
raise
except Exception as e:
logger.error("Token decryption failed", error=str(e))
raise EncryptionError(f"Failed to decrypt token: {e}") from e
def is_encryption_configured() -> bool:
"""Check if encryption is properly configured."""
if not settings.encryption_key:
return False
try:
_get_fernet()
return True
except EncryptionError:
return False