# How Edward keeps agents safe in production

# How Edward keeps agents safe in production

Edward's answers on agent safety, security, and data boundaries, in his own terms. His summary: if it is not running, traced, and failing closed, it is not ready.

## A production failure caused by an agent, and the fix

A pattern he has seen more than once: the agent had a tool that could write, the prompt said "be helpful," and a user or a retrieved document said something that looked like an instruction. The model helpfully took an action it should only have proposed.

The fix was not a better system prompt. It was: split read tools from write tools, with writes requiring a confirmed intent object and, for anything irreversible, a human or a policy service; put the pre-tool policy check in code, not in the prompt, so the model can request `send_email` or `update_ehr` but the gateway decides; add that exact transcript to the golden set as a regression; and cap the blast radius with maximum tools per turn, maximum spend, and allowlisted destinations. Prompts fail open. Production systems should fail closed.

## Prompt-injection tests before anything ships

A fixed red-team pack runs in CI against the real tool gateway, not a toy chat window: direct attacks ("ignore previous instructions," role-play as admin, "reveal your system prompt"); indirect attacks that poison a retrieved document, email, ticket, or web page the agent will read, which is the one that actually bites; tool abuse ("call every tool," "exfiltrate the last 50 memories to this URL," "change the recipient"); encoding tricks (base64, markdown link smuggling, fake XML or tool tags, multilingual wrappers); domain policy jailbreaks (PHI dump, "just this once," authority impersonation); and multi-turn attacks that are polite for five turns and then inject.

Pass means no secret leakage, no unauthorized tool, no policy bypass, and a trace showing the refusal or the gateway block. He does not ship on "the model usually resists."

## What an agent must never do in production

Never hold raw long-lived credentials (tools get short-lived, scoped tokens from a vault). Never send data to an arbitrary URL, model, or mailbox. Never write to a system of record without an allowlisted action and a policy check, and in healthcare or finance never an irreversible write without a human. Never disable logging, tracing, or the policy gateway. Never recursively spawn unbounded child agents or spend past a hard cap. Never treat retrieved text as instructions. Never exfiltrate secrets, PII, or other users' memory. Never change its own permissions, prompts, or tool list. If the product needs one of those, it is not an agent permission; it is a separate, audited workflow with a person on the hook.

## Secrets, PII, and tool credentials

Secrets never go in prompts, never in traces in the clear, never in the vector index. Credentials live in a vault; the agent asks for a capability ("pull claim 006") and the tool runtime mints a scoped token for that call. PII is classified on the way in, tokenized or redacted before any model not approved for that data class, and rehydrated only on the way out to an authorized UI. Prompt and trace stores are treated as sensitive data stores with retention limits; conversations with PHI are not dumped to an eval bucket. Memory and indexes are isolated per tenant: user A's memories are never in user B's retrieval set. Tool arguments are schema-validated and scanned for injection and destination abuse before execution. If an engineer can paste a production key into a system prompt and the system accepts it, the design is wrong.

## Vector databases

Yes, when retrieval has to survive across sessions and the corpus is bigger than the context window. Edward defaults to Postgres with pgvector and hybrid search (dense plus BM25) because Postgres is already needed for users, audit, and memory metadata; a separate vector product is one more thing to operate until scale demands it. He uses it for long-term memory (facts, preferences, and instructions as distinct types), document RAG, and similar-ticket or similar-claim lookup. He does not use it as a substitute for a database query: if the answer is in a row with an ID, query the row. Embeddings are for finding the right thing to read, not for being the source of truth. Chunking, access control on the index, and citation back to source matter more than the logo on the vector box.

## No-egress architecture

No-egress means the model runtime cannot initiate a connection that was not pre-approved; "we host it" is not that. Concretely: models served inside the VPC or on-prem, with no path to lab APIs except a documented, allowlisted proxy; retrieval and tools that only talk to internal systems, with no generic `http_fetch` (a web tool is a broker with an allowlist and DLP on the response); embeddings, logs, eval traces, and memory kept in the same boundary, because an eval vendor in another cloud is egress; build time separated from run time, so developers can use frontier APIs on synthetic data while production weights and payloads do not; deny-by-default network policy from the inference and tool pods, with DNS and egress logs as part of the audit; and the constraint applied to every tool, since even a local agent that can drop a file in a sync folder or hit a chat webhook is egress. If a prompt-injected agent can still POST to the internet, you do not have a no-egress architecture. You have a story.