mirror of
https://github.com/open-gitagent/langship.sh.git
synced 2026-08-03 07:21:04 +02:00
patel-lyzr
b6647a537d
- Implemented a new Gateway page that provides a uniform ingress for deployed agents. - Created a Runs page that lists recent pipeline executions with auto-refresh and error handling. - Added a RunsListener component to handle real-time notifications for new runs via SSE. - Updated the AppSidebar to include links to the new Gateway and Runs pages. - Enhanced FlowNode component to improve status display and styling. - Introduced EmptySection component for consistent empty state presentation. - Updated API utility to include a new endpoint for the runs stream.
131 lines
5.8 KiB
Markdown
131 lines
5.8 KiB
Markdown
# flow
|
|
|
|
A durable agent-pipeline runtime — the engine + control plane behind
|
|
[Langship](./aude.md) (Deployment / Governance / Operations for agent apps).
|
|
|
|
- **Pipeline canvas**: drag-and-drop CI/CD nodes (Trigger → Build → Test →
|
|
Eval → Policy → Approval → Deploy → Promote → Rollback)
|
|
- **n8n-shaped JSON** as the on-disk pipeline format; flow's own DAG +
|
|
executor catalog at runtime
|
|
- **Restate-backed durability**: every node wrapped in `restate.Run` —
|
|
crash-safe journaling, replay, awakeable-based human approvals
|
|
- **Real BuildKit OCI builds** with private-repo PAT support; pushes to
|
|
GHCR or any registry
|
|
- **GitHub webhook receiver** with HMAC verification, agent ↔ pipeline
|
|
attachments, branch filtering
|
|
- **Live execution view**: SSE-streamed per-node status + per-node log
|
|
lines (BuildKit progress, shell stdout, stub events), canvas-overlay
|
|
status rings
|
|
|
|
## Status
|
|
|
|
Pre-v0.1. Working but moving fast — APIs and node types may change.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌────────────────┐ ┌────────────────┐
|
|
│ web (nginx) │ │ flow API (Go) │ ┌──────────────┐
|
|
│ Next static │◄──►│ :8090 │◄────►│ Mongo │
|
|
│ :3000 │ │ /api/* CRUD │ │ pipelines │
|
|
└────────────────┘ │ /api/.../stream │ runs │
|
|
│ SSE (live) │ │ agents │
|
|
└────────┬───────┘ └──────────────┘
|
|
│
|
|
│ ingress.Send ┌──────────────┐
|
|
├──────────────►│ Restate │
|
|
│ │ :8081 ing │
|
|
│ (executes via │ :9070 admin│
|
|
│ callback) └──────┬───────┘
|
|
│ │
|
|
│ ┌────────────────┘
|
|
▼ ▼ workflow callback
|
|
┌──────────────────────┐
|
|
│ flow service :9080 │ walkDurable + executors
|
|
│ Trigger/Build/Test/ │ → BuildKit (tcp:1234) for OCI
|
|
│ Eval/Policy/Approve │ → Registry (tcp:5000) for push
|
|
│ /Deploy/Promote/RB │
|
|
└──────────────────────┘
|
|
```
|
|
|
|
## Quickstart
|
|
|
|
```sh
|
|
docker compose up
|
|
# UI: http://localhost:3000
|
|
# API: http://localhost:8090
|
|
# Restate: :8081 ingress, :9070 admin
|
|
# BuildKit: 127.0.0.1:1234
|
|
# Registry: 127.0.0.1:5050 (host port; buildkitd pushes to registry:5000 internally)
|
|
# MinIO: 127.0.0.1:9000 (S3 API), :9001 (console; minio / minio12345)
|
|
```
|
|
|
|
The base compose now bundles every service flow needs: **mongo, restate,
|
|
buildkitd, registry, minio, flow, web**. If a sibling stack already owns
|
|
one of those host ports (e.g. another langship-* set), stop that
|
|
container or override the port mapping in a `compose.override.yml`.
|
|
|
|
## Dev (hot reload)
|
|
|
|
Three terminals:
|
|
|
|
```sh
|
|
# 1) backing services
|
|
docker compose up -d mongo restate
|
|
# (and buildkitd/registry from the standalone overlay or the sibling stack)
|
|
|
|
# 2) Go API with air (rebuilds on .go changes)
|
|
make watch
|
|
|
|
# 3) Next dev server with HMR; /api proxies to :8090
|
|
make dev
|
|
```
|
|
|
|
Open `http://localhost:3000`.
|
|
|
|
## Env vars
|
|
|
|
The flow process (`./bin/flow serve` or `make watch`):
|
|
|
|
| Var | Default | Notes |
|
|
|---|---|---|
|
|
| `FLOW_ADDR` | `:8090` | API listen address |
|
|
| `FLOW_CORS_ORIGINS` | `*` | CSV allowlist |
|
|
| `FLOW_PUBLIC_URL` | (empty) | Externally-reachable base URL — used to render webhook callback URLs. Set to your `cloudflared` tunnel for GitHub webhooks. |
|
|
| `MONGO_URI` | (required) | e.g. `mongodb://localhost:27017` |
|
|
| `MONGO_DB` | `flow` | |
|
|
| `RESTATE_INGRESS_URL` | `http://localhost:8081` | |
|
|
| `RESTATE_ADMIN_URL` | `http://localhost:9070` | |
|
|
| `RESTATE_SERVICE_ADDR` | `:9080` | Service-endpoint listen addr |
|
|
| `RESTATE_DEPLOYMENT_URI` | `http://host.docker.internal:9080` | How Restate reaches us. In docker-compose this is overridden to `http://flow:9080`. |
|
|
| `BUILDKIT_HOST` | `tcp://127.0.0.1:1234` | BuildKit gRPC. In docker-compose: `tcp://buildkitd:1234` (or `host.docker.internal` when buildkitd is external). |
|
|
|
|
## Concepts
|
|
|
|
- **Agent** — a registered git repo (URL + PAT). One-click GitHub webhook
|
|
install; `/webhooks/github/{id}` verifies HMAC and dispatches runs on
|
|
push. Agents attach to pipelines.
|
|
- **Pipeline** — a DAG of nodes built on the canvas (n8n-shape JSON
|
|
underneath). Saved to Mongo; loaded fresh per run.
|
|
- **Run** — one execution of a pipeline. Restate journals each node
|
|
(`restate.Run("node:<name>", fn)`). Terminal status is written back to
|
|
Mongo's `runs` collection.
|
|
- **Live view** — `/executions/view?id=…` subscribes to
|
|
`/api/executions/{id}/stream` (SSE) for `node_started`,
|
|
`node_completed`, `node_error`, **`node_log`**, and `done` events.
|
|
|
|
## Build node
|
|
|
|
Two modes:
|
|
|
|
- `mode: "docker"` — BuildKit solves the Dockerfile against the cloned
|
|
repo and pushes to a registry. Auth: GHCR uses the agent's PAT
|
|
(`write:packages`); `localhost:*` / `registry:*` are anonymous +
|
|
insecure. Streams BuildKit's plain-mode progress as `node_log` events.
|
|
- `mode: "shell"` — escape hatch, runs `/bin/sh -c <command>` in the
|
|
cloned repo. Stdout/stderr line-streamed to the log channel.
|
|
|
|
## License
|
|
|
|
Apache 2.0
|