Private by Design, or Public by Accident
I have seen architectures labelled “private” because one model endpoint had public access disabled. The same diagram contained public storage, an unrestricted build agent, broad outbound internet and a tool calling an external API with a static secret.
Privacy is a path property. One private endpoint cannot purify an otherwise public system.
Trace the complete request
Begin with the caller and follow every hop:
user → application gateway → application → agent runtime → model → tools → data stores → observability.
For each hop, record identity, protocol, DNS resolution, network route, encryption, authorisation and logged data. Include control-plane access and deployment paths, not only runtime traffic.
If the agent can invoke tools, trace each tool independently. Tool traffic may follow a different route from the agent's own outbound traffic.
Identity before secrets
Use managed identities and RBAC wherever the platform supports them. Separate control-plane roles from data-plane roles. Scope identities per workload and environment. A production agent should not share a broad identity with a notebook, CI pipeline and developer workstation.
Secrets that remain belong in Key Vault with rotation, access logging and an owner. They do not belong in Terraform variables committed to Git, prompt text or copied connection strings.
Private endpoints require DNS architecture
Creating a private endpoint is the beginning. Clients must resolve the service name to the private address from every relevant network. Private DNS zones need links, forwarding and ownership. Hub-and-spoke networks need explicit resolution paths. Build agents and operational tooling need access without opening public exceptions that become permanent.
Test resolution from the actual runtime. A green portal icon is not evidence.
Egress is where agents become interesting
An agent is designed to reach outward through tools. That makes outbound governance central.
Use approved endpoints, firewall policy, service tags where appropriate and explicit proxy patterns. Separate read-only tools from mutating tools. Log the destination, caller, tool, action and result without dumping sensitive prompt content into every diagnostic sink.
“No public egress” is powerful only when required dependencies have a deliberate private route.
State and telemetry are data systems
Conversation state, uploaded files, vector indexes, traces and evaluation datasets may contain the most sensitive parts of the interaction. Give them retention policies, encryption, access controls, deletion processes and residency decisions.
Observability must help investigate an incident without becoming a second uncontrolled copy of user data.
Test the negative path
Security tests should prove that unauthorised routes fail:
- a user outside the group cannot retrieve the document;
- a workload subnet cannot resolve a public service address;
- an agent identity cannot call an unapproved write operation;
- a developer role cannot publish to production;
- a disabled private connection cannot silently fall back to public access.
An architecture is private when the wrong paths are closed and the correct paths are explainable.
Anything else is private by diagram and public by accident.
Begin with data classes, not subnets
Network design follows the data. Classify what the system will receive and create: public prompts, employee conversations, customer records, source code, credentials, uploaded documents, vector embeddings, traces, evaluation datasets and generated actions.
For each class, decide allowed regions, retention, encryption, operators, model routes, logging and deletion. An embedding is not automatically anonymous. A trace can reveal the question, retrieved evidence, tool arguments and business outcome. A test dataset built from incidents may be more sensitive than production telemetry.
Now the network requirements become concrete. You know which traffic must remain private, which third-party calls are prohibited and which diagnostic fields require redaction.
The control plane is part of the attack surface
Architectures often secure runtime requests and leave deployment authority broad. A developer with permission to change an agent’s connections, tools or model can alter the effective security boundary without touching the application code.
Separate roles for resource administration, model deployment, project development, connection management and production publishing. Use protected pipelines and short-lived workload identity. Record who changed the tool catalogue, retrieval index, prompt, policy and model route.
The question is not only “Who can call the agent?” It is “Who can redefine what the agent is?”
Private connectivity still needs availability design
Private endpoints, firewalls and central DNS introduce dependencies. If DNS forwarding fails, the application may lose model or storage access. If the firewall cannot resolve a required endpoint, agent tools can stall in ways that look like model failure. A build agent outside the network may lead a team to create a permanent public exception.
Treat connectivity components as workload dependencies with monitoring, capacity and recovery. Test name resolution, route, TLS and identity from every runtime and deployment environment. Decide whether an unavailable private dependency fails closed, queues work or offers a degraded read-only path.
Security that nobody can operate will eventually be bypassed.
Design egress as a catalogue
Agents make outbound calls dynamic, but egress policy does not have to be vague. Create an approved destination catalogue with owner, purpose, protocol, data classes, authentication and expected volume. Route through controlled gateways or firewalls where the platform supports it.
Separate destinations used by the application from those reachable by model-selected tools. A weather API and a production payment API should not occupy the same trust category merely because both use HTTPS.
Inspect DNS and HTTP metadata without indiscriminately logging sensitive bodies. Alert on new destinations, unusual volume and denied attempts. Make adding a legitimate endpoint fast enough that teams do not solve governance through exceptions.
Tenant isolation must survive every layer
Multi-tenant systems need tenant context in identity, retrieval, state, cache, memory, tools and observability. A filter added to the final search query is not enough if the cache key ignores tenant, a shared agent memory stores user details or a tool uses a cross-tenant service credential.
Test with two tenants that have deliberately similar document names and customer IDs. Attempt retrieval, conversation continuation, tool calls and trace access across the boundary. Test administrators separately; elevated support access should be explicit, time-bounded and audited.
Isolation is proved by negative tests.
Privacy has an operating lifecycle
Before launch, perform threat modelling and data-protection review. During delivery, scan infrastructure and code, verify policy and evaluate agent behaviour. In production, monitor access, egress, prompt injection, tool anomalies and data retention. At decommissioning, delete indexes, files, state, caches, identities, secrets and diagnostic copies.
The lifecycle matters because AI systems accumulate context. A pilot that begins with harmless documents can quietly become a repository for sensitive decisions. Controls must follow the system as its knowledge and authority grow.
Private design is not a single architectural pattern. It is the discipline of ensuring that identity, network, data and operations tell the same story.