DEV Community

Cover image for Architecting an Enterprise Network on AWS Cloud WAN

Architecting an Enterprise Network on AWS Cloud WAN

How to design a multi-region, multi-account AWS network on Cloud WAN, from segments and attachments to single-hop inspection, and the lessons from running one in production.

Most enterprise AWS networks grow the same way. One VPC becomes fifty. One region becomes three or four. A Transit Gateway appears in each region, they get peered, and a team spends a growing share of its week keeping route tables in agreement across all of them. It works, and it gets a little more brittle every quarter.

AWS Cloud WAN is the answer to that pattern. I have run it in production for a network of a few hundred accounts and several hundred VPCs across four regions, and this post is the guide I wish I had at the start: how regions connect, how segments carve up the network, how inspection works, how to attach VPCs at scale, and what to expect operationally. It is a design guide, not a tutorial. There is no console clicking here.

The problem Cloud WAN solves

A Transit Gateway is regional. Run in four regions and you run four of them, peer them in a mesh, and keep the route tables on all four telling a consistent story. Every new VPC is an attachment, a route table association, propagation into the right tables, and often a static route or two. Every new environment or partner multiplies that.

Two problems dominate at scale.

Routing intent is spread everywhere. No single document says "test can reach shared services, and nothing else." That intent lives in route tables in four regions, and you prove it by reading all of them.

Inspection is double-hop. Traffic between VPCs in different regions goes through a firewall leaving the source region and another entering the destination region. Stateful firewalls need to see both directions of a flow, and with independent regional Transit Gateways the only way to guarantee symmetry is to inspect at both ends. Twice the traversals, twice the logs, two rule sets that must agree.

Cloud WAN removes both problems, and that is the reason to adopt it.

Cloud WAN in one paragraph

Cloud WAN gives you a global network, which holds a core network that AWS operates for you. The core network has an edge in each region you choose, and AWS connects those edges over its backbone. You attach VPCs, Transit Gateways, Direct Connect gateways and VPNs to the edges. Every attachment lands in a segment, a routing domain that spans all edges. The whole thing, every segment, every rule about which segment can reach which, and where traffic gets inspected, is described in one versioned JSON document: the core network policy.

If Transit Gateway is a router you configure, Cloud WAN is a WAN you describe.

Reading the diagram

  • Columns are regional edges. Everything inside a column is attached to that edge.
  • Rows are segments. A segment exists on every edge, so cards in the same row reach each other across regions for free.
  • Green links are the shared services hub reaching into each environment. Two-way, never transitive.
  • Dashed orange arrows are internet egress. Every VPC drains to its region's inspection VPC, which holds the NAT and internet gateway.
  • Solid orange paths are two production flows to another cloud, inspected once each and exiting through the nearest Direct Connect landing.
  • The VPNs have no arrows because their path is the same as the orange one, with the VPN in place of the gateway.

The rest of the post explains each of these in turn.

Regions: edges, not peerings

Each region you list in the policy becomes a core network edge. AWS peers every edge to every other edge. You never create an inter-region peering attachment, and you never maintain inter-region routes. When a VPC attaches in Sydney, its CIDR is learned by the edges in Virginia and Oregon automatically, inside whichever segment it joined. Inter-region routing is just intra-segment routing.

The default behaviour is region affinity. Traffic stays in its home region wherever it can and crosses the backbone only when the destination or a policy says so. That gives predictable paths by default, and you override it deliberately when you want a particular region to handle something, such as inspection or egress.

An edge is a router you never see. Every attachment in a region terminates on it, it holds one route table per segment, and AWS peers it to every other edge. It has no internet gateway and no firewall of its own. Anything that needs either has to be a VPC you attach to it, which is why the inspection VPCs carry both jobs below.

Each edge needs its own ASN from a range you reserve in the policy. Pick a private range, keep it out of anything you peer with, and never change it afterwards.

Segments: the routing intent, written down

Segments are where the design lives. A segment is a routing domain that exists on every edge. Attachments in the same segment can reach each other across regions. Attachments in different segments cannot, unless the policy shares routes between them.

An enterprise segment model that works well has four groups.

Group Purpose Isolation
Environment One segment each for test, staging and production workloads Isolated. A workload segment sees itself and shared services, nothing else.
Shared services Central logging, DNS, egress, platform tooling Shared into every environment segment across all regions.
External VPNs, SaaS platforms and partners that do not fit the internal pattern One per environment. Isolated, so external parties never see each other.
Inter-cloud Footprints in other public clouds One per environment. Isolated, routed through inspection.

Three rules make this model hold.

Shared services is a hub, not a bridge. Every environment segment can reach shared services, but sharing is not transitive. Test cannot reach production by going through shared services. The policy shares one segment's routes into another explicitly, and you only share in one direction: into the hub.

Split external and inter-cloud by environment. One external segment is simpler. Three means a test VPN endpoint can never be a path into production, and inspection rules for each environment stay separate.

Use segment isolation for anything third-party. Isolation is a segment attribute. Set it and attachments inside the segment cannot reach each other, only what you share in. Every partner VPN becomes an island that reaches exactly what you routed to it.

Written out, the reachability of the whole network fits in one table. This is what the policy encodes, and it applies identically on every edge.

Segment Can reach Isolated Inspected
Test Test and shared services, all regions Yes Egress only
Staging Staging and shared services, all regions Yes Egress only
Production Production and shared services, all regions Yes Egress only
Shared services Every environment segment, all regions No Egress only
External (per environment) Its own environment, via inspection Yes Every flow
Inter-cloud (per environment) Its own environment, via inspection Yes Every flow

The segment count stays small. Twelve to fifteen segments covers most enterprises. If you find yourself creating a segment per team, you are rebuilding route tables under a new name.

Attachments: make joining the network boring

At a few hundred accounts, attaching a VPC has to be self-service or your network team becomes a ticket queue. The pattern:

  1. Share the core network to every member account with AWS Resource Access Manager. One share, one organisational unit.
  2. Let the VPC owner create the attachment in their own account, through whatever internal platform or Terraform module they already use.
  3. Have the attachment carry a tag naming its segment. The tag is the only thing the owner controls.
  4. Write an attachment policy in the core network policy that matches the tag and places the attachment in the segment. No human chooses a segment for a VPC.
  5. Let route propagation do the rest. The VPC's CIDR appears in the segment's route table on every edge.

Turn attachment acceptance off for internal segments. If your network team manages both the core network and the accounts that attach to it, a manual approve step adds friction and no security, because segment placement is already decided by the policy, not by whoever created the attachment. Keep acceptance on for external segments where a third party could be on the other end.

Enable appliance mode on attachments to inspection VPCs, and only those. Appliance mode pins both directions of a flow to the same availability zone, which stateful firewalls need. Workload VPCs do not need it and it costs you a cross-AZ hop if you turn it on everywhere.

Inspection: one hop, not two

An inspection VPC is a VPC with a stateful firewall in it, attached to the core network, that traffic is steered through. Cloud WAN steers that traffic with network function groups.

You put inspection VPCs into a group, then in the policy you say which segment-to-segment flows must go via that group. That is a send-via rule, and it is how you inspect east-west traffic between environments, or between an environment and a partner. There is also send-to, which sends a segment's traffic to a group, typically a default route, and is how you do centralised egress.

Egress deserves a sentence of its own because it surprises people. Every segment, shared services included, gets a send-to rule that installs a default route towards the group, and the inspection VPC holds the NAT and internet gateway. Nothing else on the network has a way out. A workload VPC with its own NAT gateway is a hole in the design, so do not build one.

The win is single-hop inspection. Because the core network sees the whole flow end to end, it can send both directions through the same firewall and keep the flow symmetric even when source and destination are in different regions. An inter-region flow crosses one firewall, not two. Fewer traversals, less log volume, one rule set to reason about, and far simpler troubleshooting.

Which region's firewall handles a cross-region flow? The policy decides. You can prefer the region closest to the source, set an explicit priority order, or override per edge. That lets you route a small region's traffic through a neighbour's firewall until the small region justifies its own, and fail over when a firewall needs maintenance.

Design the groups per environment. One group for production inspection, one for lower environments, one for egress. Sharing a firewall between production and test saves money and blurs a boundary you spent the segment model building.

Hybrid and multi-cloud: same edges, same segments

Everything that is not a VPC attaches to the same core network and lands in a segment like anything else.

  • Other clouds connect over a Direct Connect gateway attachment. Many enterprises deliver those circuits through a connectivity provider rather than dedicated fibre at every site. Land them in the inter-cloud segments and send all their traffic via inspection. Land the same cloud in two regions. Both gateways advertise the same prefixes into the segment, each edge prefers the nearer one, and if a circuit drops its routes withdraw and every edge converges on the other. The two gateways never talk to each other, and a VPC in Oregon does not cross the Pacific to reach a cloud it can exit to locally.
  • VPNs and partners land in external segments, one per environment, each isolated so partners never share a routing domain.
  • Existing Transit Gateways can attach as route table attachments. That is a bridge, not a native attachment type, and it is how you keep older things running while you migrate them.

The result is one place to look. A partner VPN, a link to another cloud and a production VPC are rows in the same attachment list, placed by the same policy.

Migrating from Transit Gateway without a big bang

Cloud WAN and Transit Gateway coexist well, and that is the migration.

  1. Build the core network beside the Transit Gateways. Peer each regional Transit Gateway to its local edge. The Transit Gateways' prefixes propagate into Cloud WAN, so Cloud WAN already knows how to reach everything.
  2. Attach VPCs without moving traffic. Each VPC gets a Cloud WAN attachment and its CIDR propagates, but its route tables still point at the Transit Gateway. Verify the routes are learned on every edge before going further.
  3. Cut over with managed prefix lists. Add Cloud WAN routes to VPC route tables using a prefix list, so a growing set of destinations moves to Cloud WAN while the Transit Gateway stays the default. You move routes per environment, per region, and roll back by editing a prefix list.
  4. Flip the default. Once a VPC's traffic is all on Cloud WAN, point its default route at the core network and remove the Transit Gateway attachment.

Go test, then staging, then production. Run Route Analyzer before and after every step to confirm paths, and run synthetic connectivity tests across attachments to catch what Route Analyzer cannot.

Operating it: what to know before you commit

The design above is the good news. These are the things that will shape your day-to-day.

Policy changes take minutes. Applying a new policy version, and rolling one back, takes minutes rather than seconds. On a core network carrying production that is a long time to watch a change land. Test in a lower environment first, and know exactly which version you are rolling back to before you press the button.

Everything is one document. That is the strength and the hazard. A mistake in one attachment rule affects every region at once. Treat the policy like production code: Terraform, pull requests, plan output reviewed, applied by a pipeline, never edited in the console.

The console diff gets noisy. A large policy does not always make a small change look small in the change view. Review the diff in source control first.

Permissions cut across accounts and services. The roles for the team that runs the core network, and for the automation that creates attachments in member accounts, take real least-privilege work. Start that with your security team early, not the week before production cutover.

Observability is on you. Cloud WAN gives you route tables, attachment state and a Route Analyzer. It does not give you flows. Ship VPC flow logs from every attachment and every inspection VPC into one queryable store. That is where you will actually see what the network is doing.

Cost is per edge and per attachment. You pay an hourly rate for each core network edge and each attachment, plus data processing. There is no per-request billing. A dormant region with an edge still costs money, so only list regions you actually use.

Is it worth it?

For a multi-region, multi-account network, yes. You replace a mesh of regional Transit Gateways and their route tables with one policy. You halve firewall traversals for inter-region traffic. Attaching a VPC becomes a tag. And "what can test reach?" is answered by a file instead of a scavenger hunt through four regions.

Cloud WAN is still young and it shows in change velocity and tooling. But the model is right. Describe the WAN, and let AWS run it.

If you are running a Transit Gateway mesh and weighing this up, I am happy to compare notes. Find me on blog.kasun.dev.

Top comments (0)