Enterprise MCP Architecture: Layers, Flows and Trade-offs

An enterprise MCP architecture has five layers: AI clients, an identity and policy layer, a gateway, MCP servers, and the systems of record behind them. Clients ask for tools, the gateway checks policy, servers translate calls into API or database operations, and logging records the whole path. Each layer should be replaceable on its own.

The five layers in plain terms

Clients. These are the AI applications: a chat assistant, an IDE agent, a scheduled workflow or your own agent code. They speak MCP and know nothing about your internal systems.

Identity and policy. Something has to say who the caller is and what it may do. In practice that is an API key per consumer, OAuth for human users, and a policy that maps each identity to allowed tools and data.

Gateway. The single endpoint clients connect to. It enforces the policy, filters the tool list, isolates credentials and logs calls. The gateway article covers its functions in detail.

MCP servers. Small services that expose tools, resources and prompts. One might wrap the CRM, another a document store, a third an internal billing API.

Systems of record. Databases, SaaS apps and internal APIs. They remain the truth; MCP is a doorway, not a new database.

Key takeaway: design each layer so the agent never holds a credential for the layer below it. If an agent can read a database password, the architecture has already failed.

Remote, local and in-perimeter

Where each piece runs is the most consequential choice. The protocol supports local servers launched by a client and remote servers reached over HTTP. For shared enterprise use, remote is the norm because it can be governed centrally; BusinessMCP’s guide on remote versus local MCP walks through the trade-offs.

Deployment	Strength	Weakness	Fits when
Local server on a laptop	Fast to start, no network	Hard to govern or audit	Individual developer tooling
Hosted remote endpoint	No servers to run, central policy	Data passes through a vendor	Business data already in SaaS tools
Server inside your VPC	Data stays in your perimeter	You own operations	Internal or sensitive systems
Hybrid	Governed hosted gateway plus in-perimeter servers	More moving parts	Mixed estates

The hybrid pattern is common. BusinessMCP’s Enterprise page describes exactly this shape: a governed endpoint on the platform, plus custom MCP servers deployed inside the customer’s VPC or on their own hardware, which BusinessMCP does not operate. It also states the platform runs on Vercel with its database in Supabase’s EU region and lists sub-processors on a privacy page. Whether that residency story is acceptable is a question for your own review.

Tools, resources and prompts

MCP servers expose three kinds of capability, and architecture should treat them differently. Tools perform actions or queries, so they need the strictest policy. Resources are read-only context such as a business snapshot or a document, which still need data-class controls. Prompts are reusable templates that shape how a model uses the tools. BusinessMCP documents all three in its pages on MCP resources and prompts. Policy that only covers tools will miss resources.

Request flow, step by step

1. The client connects to the gateway with a key or an OAuth token.
2. The gateway resolves the caller’s policy and returns only the permitted tools.
3. The model chooses a tool and the client sends the call.
4. The gateway checks it against policy again at execution, because the listing alone is not a control.
5. The server runs the operation using credentials the agent never sees.
6. The result passes back through the gateway, which removes fields the policy forbids.
7. The call, its arguments and the result are logged.
8. If the action is sensitive, execution pauses until a person approves.

Steps 4 and 6 are the ones teams skip. Filtering the tool list is good, but a model can still attempt a call it was never shown. Enforcing at execution and at output closes both holes. The access control article explains how data classes make this manageable.

Design trade-offs worth deciding early

One endpoint or many. One endpoint simplifies governance; many reduce blast radius. A common compromise is one gateway per environment, such as production and staging, with separate keys.

Fail open or fail closed. If the policy engine errors, should calls proceed? For enterprise use, closed is the defensible default. BusinessMCP states its policies fail closed.

Tenant isolation. Multi-tenant platforms should enforce isolation in the database, not only in application code. BusinessMCP’s security guide describes row-level security so one workspace’s key cannot read another’s data.

Model neutrality. Because MCP is an open protocol, the architecture should work with several model vendors. Check that a platform supports bring-your-own model keys if that matters to you.

Capacity and operations

Do not forget the boring parts. Plan rate limits per key, timeouts on slow tools, and a way to disable a single tool quickly. Decide who is on call when the endpoint misbehaves; BusinessMCP is explicit that its Enterprise engagement is founder-led and does not include 24/7 on-call, a detail that belongs in your risk register. Pair the architecture with the rollout steps in how to implement MCP in an enterprise.

What to put in the architecture decision record

Write the choices down while they are fresh; the people who made them will move on. A one-page record should capture the following.

• Scope. Which systems are exposed through MCP in phase one, and which are explicitly excluded.
• Placement. Which components run on a vendor platform, which in your perimeter, and which in each region.
• Identity. How agents and humans authenticate, how keys are issued and how they are revoked.
• Policy model. Whether rules are written per tool or per data class, and who approves changes.
• Failure behaviour. What each layer does when the next one is down or returns an error.
• Evidence. What you log, for how long, and who may read it.

Most production incidents in agent systems trace back to an unstated assumption in one of those six rows. The vendor comparison and the security checklist are useful companions when filling it in.

Starting small without painting yourself into a corner

You do not need the full five-layer picture on day one. A sensible first milestone is one gateway, one or two read-only servers and a single policy per agent. Because every layer speaks the same open protocol, you can swap a server or move it into your perimeter later without rewriting the clients. That is the real architectural benefit of MCP: the seams are standard, so decisions stay reversible.