# Open District — Agent Instructions

## Section Index

| # | Section | Anchor |
|---|---------|--------|
| 1 | [Product Summary](#product-summary) | `#product-summary` |
| 2 | [MCP Endpoint + Config](#mcp-endpoint--config) | `#mcp-endpoint--config` |
| 3 | [MCP Tools (10)](#mcp-tools) | `#mcp-tools` |
| 4 | [REST/API Endpoints](#restapi-endpoints) | `#restapi-endpoints` |
| 5 | [Connector Catalog](#connector-catalog) | `#connector-catalog` |
| 6 | [Event Store Facts](#event-store-facts) | `#event-store-facts` |
| 7 | [Audit/Governance Surface](#auditgovernance-surface) | `#auditgovernance-surface` |
| 8 | [Canonical Examples](#canonical-examples) | `#canonical-examples` |
| 9 | [Pricing + Signup](#pricing--signup) | `#pricing--signup` |

---

## Product Summary

Open District is data for AI that shows its work. It connects open source public data beside enterprise systems and personal context in one secure platform. Every fact has a source, every answer has citations, every action is audited. Includes: append-only event store (Postgres + ClickHouse mirror), 34 source connector templates (REST manifests, OAuth, webhooks, direct DB), federated SQL with read-only gate, semantic + lexical search (Qdrant, 1024-dim), entity graph with cited paths, pipeline engine (scheduled DAG jobs, sandboxed WASM UDFs), MCP server (10 tools), append-only audit log, SSO (OIDC), PASETO sessions, scoped API keys. Free forever. Sign up at https://opendistrict.org/#signup.

---

## MCP Endpoint + Config

- **URL:** `https://mcp.opendistrict.org/mcp`
- **Transport:** Streamable HTTP (JSON-RPC 2.0)
- **Server:** `serverInfo.name` = `udp-mcp`, `protocolVersion` = `2025-06-18`. The server returns that
  protocol version verbatim on `initialize`; it does **not** negotiate — send any version, get `2025-06-18` back.
- **Methods:** `initialize`, `ping`, `tools/list`, `tools/call`, `prompts/list`, `prompts/get`
- **Capabilities:** `tools`, `prompts`, `logging`
- **Auth:** Bearer token via scoped API key issued in-app at `app.opendistrict.org` (`POST /api/keys`)
- **Discovery:** well-known manifest at `https://opendistrict.org/.well-known/mcp-server`; hyperscaler integration guides under `/guides/` (AWS AgentCore Gateway target, Azure Foundry IQ knowledge source)

### API-key scopes

These seven strings are the **complete** set `POST /api/keys` accepts. Any other value is rejected
`400 invalid scope: <value>` — so a key requested with a scope name not listed here cannot be minted
at all. Authoritative source: `create_api_key`'s `VALID_SCOPES` in
`backend/crates/api-gateway/src/handlers.rs`, snapshotted to `ci/mcp-protocol.snapshot.json`.

| Scope | Grants |
|---|---|
| `data:read` | `get_schema`, `catalog`, `metrics`, `list_sources`, `list_pipelines`, `get_topology`, `entity_path` |
| `query:run` | `run_sql` |
| `vectors:search` | `vector_search`, `unified_search` |
| `events:write` | write events into the tenant store (not reachable through MCP — the MCP surface is read-only) |
| `pipeline:run` | trigger pipeline runs (not reachable through MCP) |
| `users:manage` | tenant user administration (not reachable through MCP) |
| `apikeys:manage` | mint and revoke API keys (not reachable through MCP) |

**For a read-only agent, mint a key with exactly `data:read`, `query:run` and `vectors:search`.**
Each `tools/call` is scope-checked individually: a key missing the required scope gets an
`isError: true` tool result reading `Error: missing required scope '<scope>' for tool '<name>'`,
not an HTTP 403.

> ⚠️ Earlier revisions of this page published the scope triple `admin`, `api_keys`, `read`.
> **None of those three strings exists** in the scope vocabulary; a key requested with them was
> rejected 400. `scripts/validate.sh` now fails if any of them is re-published. <!-- retired-names-ok -->

```json
{
  "mcpServers": {
    "opendistrict": {
      "url": "https://mcp.opendistrict.org/mcp",
      "headers": {
        "Authorization": "Bearer $OD_API_KEY"
      }
    }
  }
}
```

---

## MCP Tools

All tools require a valid Bearer token. All reads are gated; all actions are audited.

**These 10 names are the complete `tools/list` response** — generated from
`backend/crates/api-gateway/src/mcp_server.rs` `tool_list()`. Calling any other name returns
MCP error `-32602 tool not found`. Four **prompts** are also exposed via `prompts/list`
(see [Canonical Examples](#canonical-examples)); prompts are not tools.

### 1. `run_sql`

Execute a read-only SQL SELECT query over the data platform. Available tables: 'events'/'pg_events' (Postgres), 'ch_events' (ClickHouse), 'audit_log'. Cross-store federation is supported via DataFusion. Governance-enforced: SELECT-only, allowlisted tables only. NOTE: 'payload' is a JSON STRING column — to read a field's value use jsonExtractString(payload, 'key') (e.g. jsonExtractString(payload, 'kismet_device_base_type')); to filter on a key use payload ILIKE '%"key"%' (pushes down). Do NOT use payload->>'key', payload->'key', or payload::jsonb — DataFusion cannot plan those. A bare LIMIT n returns the n most-recent rows (fast, index-backed); avoid ORDER BY created_at ... LIMIT over raw rows — that ordering cannot be pushed down and forces a full-table sort; use a bare LIMIT instead.

```
run_sql(sql: string)
```

| Param | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| sql | string | yes | — | A read-only SELECT query (e.g. SELECT event_type, count(*) FROM events GROUP BY event_type LIMIT 10) |

### 2. `vector_search`

Semantic search over events using vector embeddings (Qdrant, mxbai-embed-large-v1, 1024-dim). Use this for meaning-based search rather than exact text matching. Returns events ranked by semantic similarity.

```
vector_search(query: string, top_k?: number)
```

| Param | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| query | string | yes | — | Natural-language search query |
| top_k | number | no | 10 | Number of results |

### 3. `get_schema`

Inspect the queryable schema — the tables and columns the run_sql tool can SELECT from. Call this first if you are unsure what data is available.

```
get_schema()
```

*No arguments.*

### 4. `catalog`

The semantic data catalog — curated descriptions of every queryable table and column, plus example questions and their SQL. Use this to learn what data exists and how to phrase queries before calling run_sql. System-level reference data (identical for all tenants). No arguments.

```
catalog()
```

*No arguments.*

### 5. `unified_search`

Federated semantic + lexical search across the whole platform (Qdrant vectors, ClickHouse events, Postgres full-text fallback). Returns heterogeneous ranked results. Prefer this over vector_search when you want broad coverage, not just embeddings.

```
unified_search(query: string, limit?: number, event_type?: string, source_id?: string, score_threshold?: number)
```

| Param | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| query | string | yes | — | Natural-language search query |
| limit | number | no | 20 | Max results |
| event_type | string | no | — | Optional event_type filter |
| source_id | string | no | — | Optional source UUID filter |
| score_threshold | number | no | — | Optional minimum score (0.0-1.0) |

### 6. `metrics`

Tenant-level summary metrics: total events, events today, active sources, active pipelines. No arguments.

```
metrics()
```

*No arguments.*

### 7. `list_sources`

List the tenant's data sources (name, connector_type, status, event_type, last_sync_at, last_error). Up to 50 rows. No arguments.

```
list_sources()
```

*No arguments.*

### 8. `list_pipelines`

List the tenant's pipelines (name, status, schedule, source, destination, total_runs, failed_runs). No arguments.

```
list_pipelines()
```

*No arguments.*

### 9. `get_topology`

Return the tenant's flow topology graph (sources/flows nodes + edges). Use this to understand how data moves through the tenant's pipelines. No arguments.

```
get_topology()
```

*No arguments.*

### 10. `entity_path`

Find how two entities are connected in the entity graph. Pass two entity NAMES (or IDs like MACs / CVE ids) — resolution happens server-side, no separate lookup call needed. Returns up to 3 paths of at most 3 hops; every hop cites its source event id and observation date; path confidence is the minimum edge score. Distinct negatives: 'could not resolve "<name>"' (name not found), 'no path within 3 hops' (both entities exist but are not connected), 'lookup timed out' (the graph query exceeded its ~2s budget).

```
entity_path(from: string, to: string)
```

| Param | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| from | string | yes | — | First entity name or id (e.g. 'Xerox Corporation', 'AA:BB:CC:DD:EE:FF', 'CVE-2024-1234') |
| to | string | yes | — | Second entity name or id |

---


## REST/API Endpoints

| Method | URL | Auth | Description |
|--------|-----|------|-------------|
| GET | `https://app.opendistrict.org/api/health` | none | Service health (public) |
| POST | `https://mcp.opendistrict.org/mcp` | Bearer | MCP Streamable HTTP endpoint |
| GET | `https://app.opendistrict.org/login` | none | SSO/OIDC login redirect |
| POST | `https://app.opendistrict.org/api/keys` | Bearer (admin) | Issue scoped API key |
| DELETE | `https://app.opendistrict.org/api/keys/:id` | Bearer (admin) | Revoke API key |

---

## Connector Catalog

**Total:** 34 source connector templates (19 featured datasets, keyless one-click install)

Authoritative source: `GET /sources/templates` on api-gateway, generated from
`backend/crates/api-gateway/src/templates.rs` `templates()` into
`ci/connector-templates.snapshot.json`. `scripts/validate.sh` asserts the total, the
per-category counts and the id list below equal that snapshot **as a set**, so this
section cannot drift silently the way the MCP tool list did.

A *template* is a pre-built starting configuration (a declarative REST manifest, or a
connector-specific config blob) that `POST /sources` stores verbatim into
`sources.config` — not a live ingestion. For what is actually syncing today, and the
licence triage behind every candidate source, see
[source-ledger.html](https://opendistrict.org/source-ledger.html).

### By category

The backend `category` field (gallery grouping). Counts must equal the snapshot.

| Category | Templates | Covers |
|---|---|---|
| connector | 14 | Bring-your-own-API starting points: REST manifests (cursor / offset / API-key / OAuth2), webhooks, JSON pull, file import, and the native SaaS connectors. |
| government | 6 | Regulatory, legislative and campaign-finance open data. |
| science | 4 | Research outputs, open science and climate/environmental catalogues. |
| social | 3 | Public social and community signals. |
| civic | 2 | Municipal and civic service data. |
| markets | 2 | On-chain and prediction-market data. |
| security | 2 | Threat intel feeds and vulnerability/botnet lists. |
| space | 1 | Space weather. |

### Every template id

Pass one of these as the template when creating a source. Machine-readable form:
`ci/connector-templates.snapshot.json`.

```
rest-cursor
rest-offset
rest-apikey-header
webhook
json-pull
github
salesforce
servicenow
workday
rest-oauth
kismet
file
urlhaus
threatfox
github-advisories
feodo-botnet
federal-register
govtrack-bills
nyc-311
defillama-chains
polymarket-markets
zenodo-records
inaturalist-observations
mastodon-trends
swpc-kindex
openlibrary-search
lichess-tournaments
chesscom-daily-puzzle
noaa-ncei-datasets
fec-candidates
fec-itemized
openfda-food-enforcement
healthdata-catalog
congress-bills
```

### Landing-page District map

The District map on [opendistrict.org](https://opendistrict.org/#sources) (registry:
[district.yaml](https://opendistrict.org/district.yaml)) is a **narrative grouping** of the
product into Public data / Enterprise / Personal wards. It is *not* 1:1 with this catalog:
it renders 29 blocks, several of which are roadmap or reach-via-generic-REST sources
rather than dedicated templates, and it omits several templates that exist. Treat this
section, not the map, as the answer to "what can I connect today".

---

## Event Store Facts

- **Storage:** Append-only event store backed by Postgres
- **Analytics mirror:** ClickHouse for analytical queries over large event ranges
- **Immutability:** Events are never updated or deleted; corrections are new events referencing originals
- **Temporal provenance:** Every fact carries source timestamp, ingestion timestamp, and source reference
- **Audit integration:** Every query against the event store writes an audit log entry
- **Schema evolution:** Events carry schema version; readers handle backward-compatible evolution

---

## Audit/Governance Surface

| Control | Implementation |
|---------|---------------|
| Audit log | Append-only; every query, every action |
| Row-level security | Postgres RLS policies per tenant |
| Scoped API keys | Seven scopes, minted in-app: `data:read`, `query:run`, `vectors:search`, `events:write`, `pipeline:run`, `users:manage`, `apikeys:manage` — see [MCP Endpoint + Config](#mcp-endpoint--config) for which tool needs which |
| Read-only gate | Federated SQL rejects write statements |
| SSO | OIDC providers |
| Session tokens | PASETO (v4.public) |
| Human-approval workflow | Governed actions require explicit approval |
| Governed write-backs | Audited, approved writes to external systems |
| Standing subscriptions | Recurring data syncs with governance |

---

## Canonical Examples

Four canonical scenarios, mirrored 1:1 from the machine-readable manifest [examples.yaml](https://opendistrict.org/examples.yaml) (source of truth; also exposed as MCP `prompts/list` templates). Asks are verbatim.

### follow-the-money

- **Ask:** Which companies named in new federal rules donated to the committees overseeing them?
- **Chain:** `search(q="final rule" · Federal Register · last 90 days)` → `entities(name=…)` → `query(SELECT committee, amount FROM fec_contributions)`
- **Cited answer shape:** rule hits count + committee donation total, citing Federal Register FR citation and FEC itemized records.

### scan-the-courts

- **Ask:** Is anyone challenging the new broadband rule?
- **Chain:** `search(q="petition for review" · Court opinions · related <FR cite>)` → `query(SELECT docket, status, next_hearing)`
- **Cited answer shape:** case count, lead docket, hearing date — docket and order-list citations.

### meeting-brief

- **Ask:** What should I know before my 2pm?
- **Chain:** `calendar(event=next)` × `email(counterparty=… · last 30 days)` × `search(q=<counterparty> · all sources)`
- **Cited answer shape:** deal state from thread, regulatory risk from Federal Register hit, suggested ask.

### thesis-watch

- **Ask:** Flag anything that changes my thesis on Acme.
- **Chain:** standing watch (`pipelines`) → `events(filter.entity_id=… · since last review)`
- **Cited answer shape:** thesis factor at risk + primary-source notice citation.

---

## Pricing + Signup

- **Price:** Free forever
- **License:** AGPL-3.0-only (repo: lab/www, LICENSE file); public records keep their original source licenses
- **Sign up:** https://opendistrict.org/#signup
- **Email:** hello@opendistrict.org
- **Sign in:** https://app.opendistrict.org/login
- **API keys:** Issued per tenant in-app with explicit scopes