Prompt injection is not a bug you patch: securing LLM features in production

Language models do not distinguish between instructions from your application and instructions embedded in the data they process. Any content that reaches the model — a support ticket, a scraped page, an uploaded document — is potentially instruction.

Design for the assumption that injection succeeds

Input filtering helps at the margins and fails as a control. The durable defence is architectural: assume the model can be persuaded to attempt anything within its reach, then constrain that reach.

Three boundaries worth enforcing

First, tool permissions belong to the user session, not the model. If a user cannot delete a record through the API, the assistant acting on their behalf must not be able to either. Second, any action with side effects — sending, purchasing, deleting, publishing — requires explicit human confirmation showing the concrete action, not a summary of it. Third, model output is untrusted input to everything downstream; rendering it into HTML or passing it to a shell reintroduces injection in a more familiar form.

Data flow is the real risk

The highest-impact findings we report are not clever jailbreaks. They are retrieval systems that return documents across tenant boundaries because the vector store was not filtered by the requesting user permissions. The model faithfully summarises data the user should never have seen, and the audit log shows a normal query.

Threat model the feature the way you would threat model an API that accepts arbitrary input from the internet — because that is what it is.