The Minimal AWS Stack for an AI Product
I keep seeing AI product architectures with 40 boxes on the diagram. EKS, SageMaker endpoints, three message queues, a service mesh nobody asked for. Then I ask what the product does and it’s a form that calls an LLM and writes a row to a database.
So here is what I actually run. One VPC, two subnets, one EC2 instance, one RDS database, one S3 bucket, one load balancer. Six pieces. That’s the whole stack for most AI products I’ve built, including AIVory itself in its early months.
Start With What the Product Actually Does
An AI product, in the boring technical sense, does three things. It takes a request, it calls a model (yours or someone else’s API), and it stores something - a user, a usage record, a generated file. That’s it. Everything else is optimization for a scale you don’t have yet.
I learned this the expensive way at a client in Cologne. Team of six, Kubernetes cluster with 12 microservices, and a product doing maybe 200 requests a day. The AWS bill was higher than the revenue. Nobody could tell me why the “recommendation service” needed its own deployment separate from the “recommendation-orchestrator” service. Nobody remembered.
You don’t need that. You need a place to run code, a place to store state, and a way for the internet to reach it safely.
The VPC and Two Subnets
One VPC. Inside it, one public subnet and one private subnet. The public subnet holds your load balancer - nothing else touches the internet directly. The private subnet holds your EC2 instance and your RDS database.
This is the one piece of “real infrastructure” discipline I don’t skip. Your database should never have a public IP. Not “restricted by security group,” not “fine for now” - never. I’ve seen this go wrong twice at client sites, both times because someone needed to debug something at 11pm and opened the database to 0.0.0.0/0 “temporarily.” Six months later it was still open.
With Architect, this part takes about 90 seconds - I drag a VPC block onto the canvas, drop two subnets in it, mark one public, and the tool wires the route tables and the internet gateway for me. I’ve done this manually with hand-written Terraform maybe 40 times over the years and it’s always the same 80 lines. Not interesting work.
One EC2 Instance, Not a Cluster
For the API layer, one EC2 instance. Not an Auto Scaling Group, not ECS, not EKS. A t3.medium or t3.large running your API process behind a process manager, in the private subnet.
Yes, this instance can fail. So can any single pod in a Kubernetes cluster during a bad rollout, and now you’re also debugging why the cluster autoscaler didn’t come back. A single EC2 instance with a health check on the load balancer and an EBS snapshot schedule survives the failure modes that actually happen to a product doing under 10,000 requests a day: OOM kill, a bad deploy, an AWS AZ hiccup.
When you outgrow this - genuinely outgrow it, meaning you’re paging yourself about capacity, not meaning you read a blog post about Kubernetes - you add a second instance and a target group. That’s a one-line change in Architect’s canvas, not a rewrite.
One honest limitation here: a single EC2 instance means a deploy has a few seconds of downtime unless you set up a blue-green swap yourself. Architect doesn’t script that part for you yet. For most early-stage AI products, a few seconds at 3am beats the operational tax of always-on redundancy.
RDS for State, Because State Needs a Grown-Up
The model call is stateless. Everything before and after it isn’t - users, sessions, usage counters, billing records. That goes in RDS. Postgres, single instance, in the private subnet, automated backups on.
I don’t run Aurora for a product under real load. Aurora is a good product but it’s solving a scaling problem you probably don’t have, and it costs more for the privilege of not having it yet. Plain RDS Postgres, one instance, with a read replica added later if you actually need one.
The EC2 instance talks to RDS over the private subnet, no internet round trip, no exposed port. This is also where most of the “why is my AI product slow” complaints I hear turn out to live - not the LLM call, the database round trip somebody put inside a loop.
S3 for Model Artifacts, Not for Everything
S3 gets one job here: storing anything too big or too binary for Postgres. Generated files, uploaded documents, fine-tuned model weights if you’re self-hosting anything. I don’t route application state through S3 just because it’s cheap and durable - a bucket full of JSON files pretending to be a database is a debugging session waiting to happen.
If your product calls an external LLM API (OpenAI, Anthropic, whoever), you probably don’t even need much S3 volume. It only really grows once you’re storing user-generated outputs at scale.
The Load Balancer Ties It Together
One Elastic Load Balancer in the public subnet, TLS terminated there, forwarding to the EC2 instance in the private subnet. This is the only door into your system from the internet, and it’s also where your health checks live - if the EC2 instance stops responding, the load balancer stops sending it traffic instead of serving errors to real users.
In Architect I connect the ELB block to the EC2 block with a drag, set the health check path, and the generated Terraform includes the security group rules that only allow traffic from the load balancer to the instance - nothing else can reach it, including me, without going through a bastion or SSM.
What This Doesn’t Do
I’ll say the honest part plainly: this stack does not autoscale, does not survive an entire AWS region going down, and does not give you blue-green deploys out of the box. If you’re building something that needs five nines or handles regulated financial data at real volume, this is a starting point, not a destination.
But most AI products aren’t there. Most are a founder or a small team trying to get from zero users to a hundred, then a hundred to a thousand, without the infrastructure bill or the on-call rotation eating the runway first. I’ve watched that Kubernetes-first instinct kill more small AI products through complexity than through outages.
Why I Use Architect for This
I built Architect because I got tired of writing the same VPC-subnet-EC2-RDS-S3-ELB Terraform by hand for every client and every side project. It’s a visual canvas - you drag the AWS resources onto it, connect them with lines the way you’d draw the architecture on a whiteboard, and it generates production Terraform underneath. EC2, RDS, ELB, VPC, EBS, S3 - the pieces this stack actually needs.
The honest limitation: it’s not a Kubernetes tool and I didn’t build it to become one. If your architecture genuinely needs a service mesh, look elsewhere. For the six-box stack most AI products actually run on, it gets me from blank canvas to applied Terraform in under ten minutes instead of an afternoon.
It’s $9.99/month or $99.99/year if you want to try it. Either way, the point of this post isn’t the tool - it’s that you probably don’t need the 40-box diagram. Six boxes get most AI products further than people expect.