External/langship.sh
1
0
Fork 0
mirror of https://github.com/open-gitagent/langship.sh.git synced 2026-08-03 07:21:04 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat: add contributing guidelines and enhance README structure

Browse Source
This commit is contained in:
patel-lyzr
2026-05-13 22:15:08 +05:30
parent dd2c0e171a
commit e7cf771b96
4 changed files with 383 additions and 207 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTRIBUTING.md
+103
View File
@@ -0,0 +1,103 @@
# Contributing to Langship
Thanks for your interest. This guide covers the dev setup, what to run
before opening a PR, and the conventions we follow.
By contributing, you agree your contributions are licensed under the
[Apache License 2.0](./LICENSE) (the project's license). No CLA or DCO
sign-off is required.
## Code of conduct
Participation is governed by [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md).
Report concerns to <khush@lyzr.ai>.
## Repo layout
| Path | What |
|---|---|
| `cmd/flow/` | the `flow` server binary (API + Restate worker entry point) |
| `pkg/api/` | REST + SSE handlers |
| `pkg/orchestrator/`, `pkg/engine/` | DAG walk, execution context, events |
| `pkg/executors/` | node implementations (Build, Push, SAST, Approval, Promote, Deploy, …) |
| `pkg/awsdeploy/` | AWS Bedrock AgentCore adapter (STS, ECR, IAM, control-plane SigV4) |
| `pkg/storage/` | Mongo-backed stores (pipelines, runs, agents, credentials, environments) |
| `pkg/secrets/` | AES-GCM seal/open keyed off `FLOW_SECRET_KEY` |
| `web/` | Next.js UI (static export) |
| `langship-cli/` | the `langship` Python CLI |
## Dev environment
Backing services (Mongo, Restate, BuildKit, MinIO, registry) come up via
docker-compose; the Go API runs on the host. See the **Quickstart** and
**Dev (hot reload)** sections in the [README](./README.md) for the exact
commands. In short:
```bash
make services # docker-compose: mongo, restate, buildkitd, registry, minio
make watch # Go API with air (rebuilds on .go change) — or `make serve` for a stable binary
cd web && pnpm dev # Next dev server, /api proxies to :8090
```
Set `FLOW_SECRET_KEY` in your environment before working on anything that
touches credentials/environments — the API refuses credential writes
without it.
CLI:
```bash
pip install -e ./langship-cli
langship login --api-url http://localhost:8090
```
## Before you open a PR
- **Go**: `go build ./...` and `go test ./...` must pass. Run
`gofmt`/`go vet` on changed files.
- **Web**: `cd web && pnpm tsc --noEmit` must pass (and `pnpm build` if you
changed anything that affects the static export).
- **CLI**: `pip install -e ./langship-cli && langship --help` should work;
smoke-test any command you touched against a local server.
- Keep changes focused — one logical change per PR.
- Update the README / CLI README / `aude.md` if you changed behavior they
describe.
## Conventions
- **Commits**: imperative subject, ≤ ~72 chars (`feat:`/`fix:`/`refactor:`
prefixes are welcome but not required). Squash noise before pushing.
- **Branches**: `main` is the trunk; branch off it. Don't push directly to
`main` — open a PR.
- **Code style**: match the surrounding code. Comments explain *why*, not
*what*. Executors emit `__<node>` summary objects on output items;
follow that pattern.
- **Errors**: wrap with context (`fmt.Errorf("deploy: %w", err)`); the API
layer maps `storage.ErrNotFound` → 404 and `storage.ErrAlreadyExists` →
409.
- **No secrets in code or commits.** Anything sensitive goes through
`pkg/secrets` and is never returned by the API.
## Adding a node executor
1. Implement `executors.NodeExecutor` in `pkg/executors/<name>.go`. Read
the trigger payload via `firstItem(inputs)` for `agentId` /
`environment` / `fromBranch`. Emit each input item with a `__<name>`
summary attached.
2. Register it in `pkg/executors/registry.go` (pass any stores it needs
via `RegistryDeps`).
3. Add a catalog entry in `web/lib/node-catalog.ts` and a `<Name>Form` in
`web/components/canvas/node-form.tsx`.
4. If it pauses (like Approval), special-case it in
`pkg/orchestrator/walk.go` so it isn't wrapped in `restate.Run` (it
calls Restate context methods directly).
## Reporting bugs
Open a GitHub issue with: what you did, what you expected, what happened,
and the relevant logs (`flow` server output + the run's node logs from
`langship runs logs <id>` or `/api/executions/{id}/logs/{node}`).
## Security
Vulnerabilities go to <khush@lyzr.ai> privately — see
[SECURITY.md](./SECURITY.md). Do not open a public issue.
README.md
+279 -94
View File
@@ -1,130 +1,315 @@
# flow
<div align="center">
A durable agent-pipeline runtime — the engine + control plane behind
[Langship](./aude.md) (Deployment / Governance / Operations for agent apps).
# Langship
- **Pipeline canvas**: drag-and-drop CI/CD nodes (Trigger → Build → Test →
Eval → Policy → Approval → Deploy → Promote → Rollback)
- **n8n-shaped JSON** as the on-disk pipeline format; flow's own DAG +
executor catalog at runtime
- **Restate-backed durability**: every node wrapped in `restate.Run` —
crash-safe journaling, replay, awakeable-based human approvals
- **Real BuildKit OCI builds** with private-repo PAT support; pushes to
GHCR or any registry
- **GitHub webhook receiver** with HMAC verification, agent ↔ pipeline
attachments, branch filtering
- **Live execution view**: SSE-streamed per-node status + per-node log
lines (BuildKit progress, shell stdout, stub events), canvas-overlay
status rings
**Any framework. Any runtime.**
## Status
Open-source, self-hosted **deployment · governance · operations** for agent apps.
One pipeline definition → Kubernetes, AWS Bedrock AgentCore, or Vertex AI Agent
Engine — same governance everywhere. Works with LangGraph, LangChain, LlamaIndex,
CrewAI, AutoGen, or raw-SDK agents. No framework lock-in.
Pre-v0.1. Working but moving fast — APIs and node types may change.
[langship.sh](https://langship.sh) · [CLI](./langship-cli/) · [Positioning](./aude.md) · Apache 2.0
## Architecture
</div>
```
┌────────────────┐ ┌────────────────┐
│ web (nginx) │ │ flow API (Go) │ ┌──────────────┐
│ Next static │◄──►│ :8090 │◄────►│ Mongo │
│ :3000 │ │ /api/* CRUD │ │ pipelines │
└────────────────┘ │ /api/.../stream │ runs │
│ SSE (live) │ │ agents │
└────────┬───────┘ └──────────────┘
│
│ ingress.Send ┌──────────────┐
├──────────────►│ Restate │
│ │ :8081 ing │
│ (executes via │ :9070 admin│
│ callback) └──────┬───────┘
│ │
│ ┌────────────────┘
▼ ▼ workflow callback
┌──────────────────────┐
│ flow service :9080 │ walkDurable + executors
│ Trigger/Build/Test/ │ → BuildKit (tcp:1234) for OCI
│ Eval/Policy/Approve │ → Registry (tcp:5000) for push
│ /Deploy/Promote/RB │
└──────────────────────┘
```
> **Repo orientation.** This is the engine + control plane. The Go service is
> codenamed `flow`; the product is **Langship**. The CLI lives in
> [`langship-cli/`](./langship-cli/).
## Quickstart
---
- [What you get](#what-you-get)
- [5 minutes to a green run](#5-minutes-to-a-green-run)
- [Architecture](#architecture)
- [Repo map](#repo-map)
- [The `langship` CLI](#the-langship-cli)
- [Concepts](#concepts)
- [Nodes](#nodes)
- [Reference — env vars & make targets](#reference)
- [Contributing & community](#contributing--community)
- [License](#license)
---
## What you get
| | |
|---|---|
| **Pipelines as graphs** | Drag-and-drop CI/CD nodes — Trigger → Build → Scan/SAST → Eval → Policy → Approval → Deploy → Promote → Rollback. n8n-shaped JSON on disk; YAML in git is the source of truth. |
| **Governance is a node** | Approvals, policy checks, eval gates, PII/secret scans are first-class, reorderable steps in the graph — not middleware you can't see. |
| **Any runtime, one pipeline** | Same definition deploys to K8s, Bedrock AgentCore, or Vertex Agent Engine. (Today the Deploy node ships to **Bedrock AgentCore** end-to-end; K8s / Vertex are stubbed.) |
| **Durable by construction** | Restate journals every node (`restate.Run("node:<name>", fn)`) — crash-safe replay, awakeable-based human approvals (timeout → auto-reject). |
| **GitOps promotion** | A Promote node opens/merges a PR `fromBranch → toBranch` on the agent's repo; the merge fires the next environment's pipeline. Promotion is an auditable event. |
| **Real OCI builds** | BuildKit solves your Dockerfile against the cloned repo, pushes to GHCR or any registry (private-repo PAT support). Mirror to N registries with the Push node. |
| **Operate, don't just deploy** | Live SSE log streams + canvas-overlay status rings; per-node logs archived to S3-compatible storage. |
| **Self-hosted, end-to-end** | Your cloud credentials, agent code, and run history never leave your network. Secrets AES-GCM sealed at rest. |
| **CLI-first** | `langship` — agents, envs, pipelines, creds, runs from your terminal. `git push` to ship. |
---
## 5 minutes to a green run
**0. Start everything.** Base compose bundles every service `flow` needs —
`mongo, restate, buildkitd, registry, minio, flow, web`.
```sh
docker compose up
# UI: http://localhost:3000
# API: http://localhost:8090
# Restate: :8081 ingress, :9070 admin
# BuildKit: 127.0.0.1:1234
# Registry: 127.0.0.1:5050 (host port; buildkitd pushes to registry:5000 internally)
# MinIO: 127.0.0.1:9000 (S3 API), :9001 (console; minio / minio12345)
```
The base compose now bundles every service flow needs: **mongo, restate,
buildkitd, registry, minio, flow, web**. If a sibling stack already owns
one of those host ports (e.g. another langship-* set), stop that
container or override the port mapping in a `compose.override.yml`.
| | URL |
|---|---|
| UI | http://localhost:3000 |
| API | http://localhost:8090 |
| Restate | `:8081` ingress · `:9070` admin |
| BuildKit | `tcp://127.0.0.1:1234` |
| Registry | `127.0.0.1:5050` (host port; buildkitd pushes to `registry:5000` internally) |
| MinIO | `127.0.0.1:9000` (S3 API) · `:9001` console (`minio` / `minio12345`) |
## Dev (hot reload)
> If a sibling stack already owns one of those host ports, stop it or override the
> mapping in a `compose.override.yml`.
Three terminals:
**1. Install the CLI and point it at the API.**
```sh
# 1) backing services
docker compose up -d mongo restate
# (and buildkitd/registry from the standalone overlay or the sibling stack)
pip install -e ./langship-cli # optional: pip install pyyaml (for -o yaml)
langship login --api-url http://localhost:8090
```
# 2) Go API with air (rebuilds on .go changes)
make watch
**2. Register an agent (a git repo) and push a pipeline.**
```sh
langship agents create --repo https://github.com/you/your-agent --pat ghp_...
langship pipelines push examples/hello.json # prints the new pipeline id
```
**3. Wire it into an environment, follow it, run it.**
```sh
langship envs create dev -d "Auto-deploy on push"
langship envs add-pipeline dev <pipelineId>
langship agents follow-env <agentId> dev
langship agents trigger <agentId> # → prints execution id(s)
```
**4. Watch it run.**
```sh
langship runs logs <executionId> -f # live SSE stream
# or open the UI: http://localhost:3000/executions/view?id=<executionId>
```
That's the loop: `agent → env → pipeline → trigger → durable run → status`.
### Hot-reload dev (three terminals)
```sh
# 1) backing services only
docker compose up -d mongo restate # + buildkitd/registry from the overlay
# 2) Go API with air — rebuilds on .go change
make watch # or `make serve` for a stable binary
# 3) Next dev server with HMR; /api proxies to :8090
make dev
```
Open `http://localhost:3000`.
`make watch` pre-exports env defaults matching the compose host ports — override
any at the CLI, e.g. `make watch MINIO_ENDPOINT=...`. Set `FLOW_SECRET_KEY` in
your shell before touching anything credential/environment-related (the API
refuses credential writes without it).
## Env vars
---
The flow process (`./bin/flow serve` or `make watch`):
## Architecture
| Var | Default | Notes |
|---|---|---|
| `FLOW_ADDR` | `:8090` | API listen address |
| `FLOW_CORS_ORIGINS` | `*` | CSV allowlist |
| `FLOW_PUBLIC_URL` | (empty) | Externally-reachable base URL — used to render webhook callback URLs. Set to your `cloudflared` tunnel for GitHub webhooks. |
| `MONGO_URI` | (required) | e.g. `mongodb://localhost:27017` |
| `MONGO_DB` | `flow` | |
| `RESTATE_INGRESS_URL` | `http://localhost:8081` | |
| `RESTATE_ADMIN_URL` | `http://localhost:9070` | |
| `RESTATE_SERVICE_ADDR` | `:9080` | Service-endpoint listen addr |
| `RESTATE_DEPLOYMENT_URI` | `http://host.docker.internal:9080` | How Restate reaches us. In docker-compose this is overridden to `http://flow:9080`. |
| `BUILDKIT_HOST` | `tcp://127.0.0.1:1234` | BuildKit gRPC. In docker-compose: `tcp://buildkitd:1234` (or `host.docker.internal` when buildkitd is external). |
Three layers, all run by you:
```
┌──────────────────────────────────────────────────────────┐
CLI ──────► API / control plane (Go — pkg/api) │
UI ──────► REST + SSE · agents/envs/pipelines/creds/runs · webhooks │
└─────────────┬────────────────────────────────────────────┘
│ RunAsync
┌─────────────▼────────────────────────────────────────────┐
│ Orchestration (Restate cluster + worker) │
│ DAG walk (pkg/orchestrator) → executors (pkg/executors) │
│ every node = restate.Run("node:<name>", fn) │
└─────────────┬────────────────────────────────────────────┘
│
┌─────────────▼────────────────────────────────────────────┐
│ Data: MongoDB (pipelines · runs · agents · creds · │
│ environments) │
│ MinIO / S3 (archived per-node logs, artifacts) │
│ Postgres — Restate's backing store ONLY │
│ pkg/secrets — AES-GCM seal/open (FLOW_SECRET_KEY)│
└──────────────────────────────────────────────────────────┘
▲
GitHub webhook │ /webhooks/github/{id} (HMAC-verified)
│ push → branch filter → dispatch run(s)
```
A run's lifecycle: webhook (or `langship agents trigger`) → the dispatcher walks
the agent's followed environments, applies each pipeline's branch filter, stamps
`agentId / environment / fromBranch` into the trigger payload, and calls
`orchestrator.RunAsync` → the DAG walker runs nodes in topological order, each
wrapped in `restate.Run` → terminal status written back to Mongo `runs` → SSE
clients (`/api/executions/{id}/stream`) get `node_started / node_log /
node_completed / node_error / done` events live.
> **Why these choices** — Restate gives crash-safe journaling + awakeables (human
> approval that survives a restart) for free; Mongo is the app store; Postgres is
> *only* Restate's persistence and is never touched by app code; BuildKit does
> real OCI builds without a Docker daemon. See [aude.md](./aude.md) for the full
> rationale.
---
## Repo map
```
cmd/flow/ the `flow` server binary (API + Restate worker entry point)
pkg/
api/ REST + SSE handlers (agents, envs, pipelines, creds, runs, webhooks)
orchestrator/ DAG walk; Approval is special-cased out of restate.Run (it
calls restate.Set/Clear directly)
engine/ execution context, ExecutionEvent, the executor lookup
executors/ node implementations + the registry:
trigger · build · push · sast · imagescan · approval ·
promote · deploy · (test/eval/policy/rollback stubs)
awsdeploy/ AWS Bedrock AgentCore adapter — STS AssumeRole, idempotent
ECR + IAM bootstrap, control-plane SigV4, endpoint wait
github/ REST helpers — webhook install/verify, PRs, merges
storage/ Mongo-backed stores: pipelines, runs, agents, credentials,
environments
secrets/ AES-GCM SealString/OpenString keyed off FLOW_SECRET_KEY
web/ Next.js 15 UI (static export) — canvas, runs, agents,
environments, credentials
langship-cli/ the `langship` Python CLI (Typer / Rich / httpx)
examples/ sample pipeline JSON
```
---
## The `langship` CLI
The daily driver for agent devs; the bootstrap surface for platform engineers.
```sh
pip install -e ./langship-cli # + pip install pyyaml for -o yaml
langship login --api-url http://localhost:8090 # saved to ~/.langship/config.toml
# the loop
langship agents create --repo https://github.com/you/agent --pat ghp_...
langship pipelines push prod.yaml --id <pipelineId> # create-or-update from a file
langship envs create prod -d "Strict gates"
langship envs add-pipeline prod <pipelineId>
langship envs reorder prod <pid1> <pid2> <pid3> # promotion order
langship agents follow-env <agentId> prod
langship agents trigger <agentId>
langship runs logs <executionId> -f
# credentials (server needs FLOW_SECRET_KEY)
langship creds create prod-aws --type aws \
--aws-region us-east-1 --aws-account 123456789012 \
--aws-role-arn arn:aws:iam::123456789012:role/FlowDeployRole
```
Command groups: `agents`, `envs`, `pipelines`, `creds`, `runs` — each with
`--help`. `-o json` / `-o yaml` on list/get commands. `LANGSHIP_API_URL` /
`LANGSHIP_TOKEN` override the saved config. Full reference:
[`langship-cli/README.md`](./langship-cli/README.md).
---
## Concepts
- **Agent** — a registered git repo (URL + PAT). One-click GitHub webhook
install; `/webhooks/github/{id}` verifies HMAC and dispatches runs on
push. Agents attach to pipelines.
- **Pipeline** — a DAG of nodes built on the canvas (n8n-shape JSON
underneath). Saved to Mongo; loaded fresh per run.
- **Run** — one execution of a pipeline. Restate journals each node
(`restate.Run("node:<name>", fn)`). Terminal status is written back to
Mongo's `runs` collection.
install; `/webhooks/github/{id}` verifies the HMAC signature and dispatches runs
on push. An agent **follows environments** (`agent.environments[]`); triggering
it runs the pipelines of every followed env. Agents may carry per-agent
credential overrides.
- **Environment** — a named, **ordered list of pipelines** (the promotion
sequence; reorderable). Global. Purely a sequencing container — per-deploy
config lives on the nodes, not the env. `dev` / `staging` / `prod` / custom.
- **Pipeline** — a DAG of nodes built on the canvas (n8n-shape JSON underneath),
stored in Mongo, loaded fresh per run. The Trigger node carries `fromBranch` /
`toBranch`; a per-pipeline branch filter decides which pipelines run for a given
push.
- **Credential** — a named record (`aws` / `gcp` / `kv`) in a global pool, with
optional per-agent overrides. Secret fields are AES-GCM sealed at rest with
`FLOW_SECRET_KEY`. Deploy / Push look one up by name.
- **Run** — one execution of a pipeline. Restate journals each node. Terminal
status is written back to Mongo's `runs` collection. The dispatcher stamps
`agentId`, `environment`, and `fromBranch` into the trigger payload; each node
emits a `__<node>` summary object on its output items.
- **Live view** — `/executions/view?id=…` subscribes to
`/api/executions/{id}/stream` (SSE) for `node_started`,
`node_completed`, `node_error`, **`node_log`**, and `done` events.
`/api/executions/{id}/stream` (SSE) for `node_started`, `node_completed`,
`node_error`, **`node_log`**, and `done` events; the canvas overlays status
rings on each node.
## Build node
---
Two modes:
## Nodes
- `mode: "docker"` — BuildKit solves the Dockerfile against the cloned
repo and pushes to a registry. Auth: GHCR uses the agent's PAT
(`write:packages`); `localhost:*` / `registry:*` are anonymous +
insecure. Streams BuildKit's plain-mode progress as `node_log` events.
- `mode: "shell"` — escape hatch, runs `/bin/sh -c <command>` in the
cloned repo. Stdout/stderr line-streamed to the log channel.
| Node | What it does |
|---|---|
| **Trigger** | Entry point; carries `fromBranch` / `toBranch` for the branch filter + Promote. |
| **Build** | Clones the agent repo (`fromBranch`), builds an OCI image via BuildKit (`mode: docker`) or runs `/bin/sh -c <command>` in the clone (`mode: shell`). GHCR auth uses the agent's PAT (`write:packages`); `localhost:*` / `registry:*` are anonymous + insecure. Streams BuildKit's plain-mode progress as `node_log` events. |
| **Push** | Mirrors the built image to one or more registries (go-containerregistry's `crane`). |
| **SAST / ImageScan** | Sibling-container scanners — trivy / semgrep / gitleaks / SonarCloud / grype — over the source / image. Configurable severity threshold and fail-on-finding. |
| **Approval** | Pauses on a Restate awakeable until resumed via `POST /api/executions/{id}/resume` (UI or `langship`). `method: ui \| quorum \| auto`; optional `timeoutSeconds` → auto-reject. Two outputs: approved (0) / rejected (1). |
| **Promote** | Opens or merges a PR `fromBranch → toBranch` on the agent's repo via the GitHub API — idempotent (re-finds an existing PR). Modes: `open-pr` / `merge` / `merge-pr`. Emits `__promote` with the PR number / URL. The merge fires the next env's pipeline. |
| **Deploy** | Deploys the upstream Push image to **AWS Bedrock AgentCore** (`target: agentcore`; k8s / vertex are stubs). Looks up an `aws` credential by name, assumes the cross-account role, idempotently provisions the ECR repo + the shared `agentcore-runtime-role` IAM role, creates/updates the runtime, waits for the endpoint to be `READY`, and emits `__deploy` with the public invoke URL. |
| **Test / Eval / Policy / Rollback** | Stubbed for now — visible on the canvas, no-op executors. |
Adding a node? See the "Adding a node executor" section in
[CONTRIBUTING.md](./CONTRIBUTING.md).
---
## Reference
### Env vars (the `flow` process — `./bin/flow serve`, `make watch`, or compose)
| Var | Default | Notes |
|---|---|---|
| `FLOW_ADDR` | `:8090` | API listen address |
| `FLOW_CORS_ORIGINS` | `*` (compose: `http://localhost:3000`) | CSV allowlist |
| `FLOW_PUBLIC_URL` | (empty) | Externally-reachable base URL for webhook callback URLs. Set to your `cloudflared` tunnel for GitHub webhooks. |
| `FLOW_SECRET_KEY` | (unset → credential writes refused) | Master key for AES-GCM sealing of credentials/secrets. Any string; hashed to 32 bytes. **Losing it makes sealed data unrecoverable.** |
| `MONGO_URI` | (required; compose: `mongodb://localhost:27017`) | |
| `MONGO_DB` | `flow` | |
| `RESTATE_INGRESS_URL` | `http://localhost:8081` | |
| `RESTATE_ADMIN_URL` | `http://localhost:9070` | |
| `RESTATE_SERVICE_ADDR` | `:9080` | Service-endpoint listen addr |
| `RESTATE_DEPLOYMENT_URI` | `http://host.docker.internal:9080` | How Restate reaches us; compose overrides to `http://flow:9080` |
| `BUILDKIT_HOST` | `tcp://127.0.0.1:1234` | BuildKit gRPC; compose: `tcp://buildkitd:1234` |
| `MINIO_ENDPOINT` / `MINIO_ACCESS_KEY` / `MINIO_SECRET_KEY` / `MINIO_BUCKET` / `MINIO_USE_SSL` | `127.0.0.1:9000` / `minio` / `minio12345` / `flow-logs` / `false` | Archived per-node log storage |
CLI env: `LANGSHIP_API_URL`, `LANGSHIP_TOKEN` (override `~/.langship/config.toml`).
### Make targets
| | |
|---|---|
| `make build` | build the web bundle then the Go binary (`bin/flow`) |
| `make build-go` | Go binary only (expects `web/dist` to exist) |
| `make serve` | `build-go` then `./bin/flow serve` — stable binary |
| `make watch` | Go API with `air` (rebuilds on `.go` change), env defaults pre-exported |
| `make dev` | Next dev server with HMR (`/api` proxies to `:8090`) |
| `make web` | build the Next static export |
| `make test` / `make vet` / `make tidy` | `go test ./...` / `go vet ./...` / `go mod tidy` |
---
## Contributing & community
- **Issues & discussion** — open a GitHub issue for bugs and feature requests. Search first.
- **Contributing** — [CONTRIBUTING.md](./CONTRIBUTING.md): dev setup, what to run before a PR, conventions, how to add a node executor. Contributions accepted under Apache 2.0.
- **Code of conduct** — [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md) (Contributor Covenant). Report concerns to <khush@lyzr.ai>.
- **Security** — **do not** file public issues for vulnerabilities. See [SECURITY.md](./SECURITY.md) — report privately to <khush@lyzr.ai>.
## License
Apache 2.0
[Apache 2.0](./LICENSE)
aude.md
-112
View File
@@ -1,112 +0,0 @@
# Langship
Langship is a framework-agnostic deployment, governance, and operations layer for agent applications. It supports LangChain/LangGraph and other agent frameworks (LlamaIndex, CrewAI, AutoGen, Pydantic AI, raw SDK agents, etc.) — not tied to any single framework.
## Three pillars
- **Deployment** — packaging, versioning, rollouts, env/secret management for agent apps
- **Governance** — policies, budgets, approvals, audit logs, tenant isolation, safety filters
- **Operations** — monitoring, replay/debug, incident response, cost & performance management
## Deployment targets
Langship is multi-runtime. Supported deployment targets include:
- **Kubernetes** (self-hosted / any cloud)
- **AWS Bedrock AgentCore Runtime**
- **GCP Vertex AI Agent Engine**
The deployment layer abstracts over these runtimes so the same agent app definition can ship to any of them. Governance and operations policies must apply uniformly across runtimes.
## Positioning
Closer to a "platform for agents" (Kubernetes/Datadog-style) than a single-framework tool like LangSmith or LangGraph Platform.
## Top-level scope — Project
Langship's top-level scoping primitive is a **Project**. A Project owns: users + roles, environments, workflows, secrets, deployment configs, audit log. Self-hosted Langship typically serves several teams, so Projects isolate one team/agent from another from day one (rather than retrofitting tenancy later). API paths are scoped: `/projects/:id/...`. Note: "Project" in this doc always refers to the Langship Project; cloud-provider projects (e.g., GCP project) are always qualified with the provider name.
## CI/CD — configurable workflow builder
CI/CD is **not** a fixed pipeline. It is a **configurable, drag-and-drop workflow builder** where pipelines are graphs of nodes (triggers, build, test, eval, policy, approval, deploy, promote, rollback). The visual canvas is the UI; YAML in git is the source of truth (GitOps).
- **Two modes** — replace CI entirely (Langship runs build/test/eval/deploy) or CD + release-gates only (external CI hands off an artifact, Langship picks up at eval/governance/deploy).
- **Role-based views** — agent devs, platform/DevOps, governance owners see different node palettes on the same underlying graph.
- **Environments are first-class** — dev, staging, prod each have their own pipeline. Per-env config (secrets, runtime, scaling) is separate from the graph. Different runtimes per env are expected (e.g., dev on K8s, prod on Vertex Agent Engine).
- **Promotion is gated and branching-strategy-driven** — evals must pass → approval gate (human, automated policy, or quorum) → `Promote` node executes per project's branching strategy (trunk-based, env-branches, release branches, or custom). Promotion and rollback are auditable events.
- **Governance is a node, not a wrapper** — policies/approvals/budget gates are first-class, visible, reorderable steps in the graph.
## Deployment model — self-hosted
Langship is **self-hosted by the customer**. The customer runs the whole stack (API server, Restate, Postgres, workers, secrets manager) in their own infrastructure. Distribution is via Helm chart / installer / Docker Compose for local dev.
**Why self-hosted:**
- Customer's cloud credentials, agent code, eval data, and audit logs never leave their network — strong fit for the governance-focused positioning
- Clear regulatory story for finance/healthcare/gov buyers who can't adopt hosted control planes
- Simpler security model: no cross-tenant credential storage, no proxy of LLM traffic
- Customer's compliance team monitors logs in systems they already operate
**Trade-offs accepted:**
- Higher friction to adopt vs. hosted SaaS — installer/upgrade UX matters more
- Support is harder — no production access by default; need good telemetry-with-consent + clear runbooks
- Distribution: ship a Helm chart as primary path; Docker Compose for local dev / small teams
**Hosted offering may come later** as a managed deployment of the same stack, but the product is designed self-hosted-first. CLI/UI/API contracts assume the server is something the customer operates.
## Server architecture — three layers
The "server" is actually three layers, all running in the customer's infrastructure:
1. **API / control plane** — REST or gRPC. CLI and UI call this. Handles auth, RBAC, workflow CRUD, run triggers, approvals, audit queries. Stateless app servers.
2. **Orchestration layer** — Restate (primary) cluster + worker pool. Workers execute node logic (build, eval, deploy, etc.). Long-lived and durable.
3. **Data layer** — MongoDB (runs, approvals, audit, users, projects, workflows index); S3-compatible object store (artifacts, large eval outputs, trace blobs); secrets manager (Vault or cloud-native KMS) for cloud credentials. Postgres runs alongside, but **only** as Restate's required persistence backend — never accessed by Langship app code.
Plus a **GitOps sync** component (initially inside the API, possibly its own service later) that watches git refs and triggers workflows on push/tag/merge events.
### Minimal v0 server
For the CLI-first vertical slice:
- One API server process, Postgres-backed
- One Restate (Restate Cloud is fine for prototyping; final product self-hosts Restate too)
- One worker process executing a few node types
- CLI talks to the API
- No UI yet
That's the smallest thing that closes the loop: `langship deploy` → API receives → Restate workflow runs → status updates → CLI shows result.
## User flow — two roles, two experiences
Langship has two distinct user journeys. The product must serve both well.
### Platform engineer — one-time project setup
Heavy, infrequent. Done once per project, occasionally revisited. Mixes CLI + UI.
1. **Environments** — define dev / staging / prod (and any custom envs like `eu-prod`, `preview`)
2. **Branching strategy** — trunk-based / env-branches / release-branches / custom
3. **CI/CD pipeline with stages** — the workflow graph (build → eval → approval → deploy → promote, etc.) per env
4. **Deployment configs** — per-env cloud credentials and runtime targets (e.g., K8s cluster X for dev, Bedrock AgentCore account Y for staging, Vertex Agent Engine project Z for prod)
Output: a configured project that agent developers consume.
### Agent developer — repeatable deploy loop
Light, frequent. Daily driver. CLI-first.
5. **Drop in agent repo** — link the repo to a Langship project (GitOps: Langship watches refs per env, not a one-time blob upload)
6. **Select** — pick project / pipeline / env target
7. **Deploy** — trigger the run; pipeline executes; agent ships
### Implications for product surface
- **CLI is the daily-use surface** for agent devs (steps 5–7) and the bootstrap surface for platform engineers (steps 1–4 initially).
- **UI is the visualization + governance surface** — workflow canvas, run history, approval inbox, audit log, release-flow view across envs. Comes after CLI proves the model.
- **Build CLI first.** A working CLI gives end-to-end usefulness sooner; UI built on top of unproven flows risks designing for the wrong thing.
- **Agent repo is git-based, not uploaded.** Langship references the repo by URL + ref; pipelines trigger on push/tag/merge events per env's branching strategy. Preserves traceability, reproducibility, and the GitOps story.
## Design principle
Core APIs and data models must stay framework-agnostic. The key abstraction is a common interface across frameworks (runs, traces via OpenTelemetry/OpenLLMetry) so governance and operations policies apply uniformly regardless of the underlying agent framework. Avoid LangChain-only assumptions in core abstractions.
Engine Restate) must stay wrapped behind Langship's own DSL — users never see the engine directly. Swapping later is possible but disruptive; pick deliberately.
claude.md
+1 -1
View File
@@ -109,4 +109,4 @@ Light, frequent. Daily driver. CLI-first.
Core APIs and data models must stay framework-agnostic. The key abstraction is a common interface across frameworks (runs, traces via OpenTelemetry/OpenLLMetry) so governance and operations policies apply uniformly regardless of the underlying agent framework. Avoid LangChain-only assumptions in core abstractions.
Engine Restate) must stay wrapped behind Langship's own DSL — users never see the engine directly. Swapping later is possible but disruptive; pick deliberately.
Engine Restate must stay wrapped behind Langship's own DSL — users never see the engine directly. Swapping later is possible but disruptive; pick deliberately.