How I Choose the Architecture for a Growing Product

The most useful system design lesson I have learned is simple: every component should solve a problem you can name.

It is easy to draw an architecture full of load balancers, caches, queues, replicas, and graphs. It is much harder, and more valuable, to explain what becomes slow, unsafe, unavailable, or expensive without each one.

I use this article as a practical framework when I am thinking through a SaaS product, an internal tool, or an AI workflow. It is not a checklist of technologies to install. It is a way to decide what to introduce next.

Start with requirements

Before choosing a database or cloud provider, write down two kinds of requirements.

Functional requirements

Describe the actions the application supports. In an order management system, that could mean creating an order, uploading a configuration, generating a report, and notifying a customer. State who may perform each action and what a successful result means.

Non functional requirements

Describe the quality and operating constraints for those actions. Specify acceptable latency, availability, access controls, and recovery objectives. A target needs a measurement window and a defined workload before it can guide a design decision.

“Handle one million requests” is not a requirement by itself. A useful requirement names the operation and constraint: for example, return a product search result within a defined latency target, retain an audit trail, or continue accepting uploads while PDF generation is delayed.

Is the problem data intensive or compute intensive?

This distinction saves a lot of wrong scaling decisions.

A real product often has both kinds of work. For example, an order dashboard may spend most of its time waiting for data while a PDF worker spends CPU time rendering. Profile the specific operation before choosing where to add capacity.

Scale in stages

Start with a single application instance and a relational database when that is enough. Then follow the bottleneck.

1. Make the code and queries efficient. Remove repeated database reads, add the index the query needs, and avoid treating browser state as an authorization decision.
2. Scale up when it is the simplest safe answer. More CPU or RAM can buy useful time without introducing distributed system complexity.
3. Scale out when one instance is the constraint. Add application instances behind a load balancer, then make sessions and file access shared or stateless.
4. Scale the database differently. Application replicas do not make a slow query faster. Investigate indexing, caching, connection limits, read replicas, or partitioning based on the actual workload.

Choose a data model for the workload

PostgreSQL is a strong default for transactional data because it gives schemas, constraints, joins, and transactions. Supabase can provide a managed PostgreSQL experience when that reduces operational work. A document, key value, columnar, or graph store can be a better fit for a specific access pattern.

The decision is not “SQL means consistency; NoSQL means scale.” Both families involve trade offs. Choose a model based on how the product reads and writes data, the consistency it needs, and how the team will operate it.

Cache deliberately

A cache is a limited, faster copy of data. It is useful when the application can tolerate a miss, has a clear invalidation strategy, and knows how stale a value may be.

• Cache aside: the application reads the cache first, then fetches and populates on a miss.
• Read through: the cache layer loads missing values from the backing store.
• Write through: a write updates the cache and backing store together.
• Write behind: a write reaches the cache first and is persisted later; this needs careful durability and failure handling.
• Write around: writes bypass the cache so recently written, rarely read data does not evict useful entries.

Redis is commonly useful for short lived data, rate limiting, and cache entries. It is not a reason to avoid a source of truth in PostgreSQL.

Move slow work out of the request path

A user should not wait for a PDF to render, a large document set to index, or every downstream notification to complete. Save the durable request state first, enqueue work, and let a worker process it.

Validate and savePersist the business record and the intent to perform background work.

Dispatch and processDeliver the job through a recoverable queue and run it in a worker.

Record the outcomeMake success, failure, and retry status visible to the application.

This is where it is important to use the right words:

• A queue distributes units of work to consumers and needs retries, idempotency, visibility, and a dead letter queue for failures that cannot be processed safely.
• Publish/subscribe distributes events to interested subscribers. Redis Pub/Sub is useful for transient fan out, but it is not a durable job queue because disconnected consumers miss messages.
• A dead letter queue, not “dead lock queue,” is where repeatedly failing messages can be inspected without blocking healthy work.

Observe the system you actually built

An architecture is only reliable when someone can tell what it is doing under failure.

• Prometheus collects and queries metrics such as request rate, error rate, latency, CPU, memory, and queue depth.
• Loki stores logs; Alloy can collect and forward logs, metrics, and traces.
• Grafana makes those signals visible in dashboards and alerts.
• Uptime Kuma is useful for an external check that a public endpoint is still reachable.

For a self hosted product, a Hetzner server with Docker and Coolify can be a sensible low complexity beginning. For systems that need managed IAM, regional services, or organisation level controls, AWS may be the better fit. Neither choice removes the need for backups, secret management, patching, and tested recovery.

Review the design against its requirements

There is no prize for adding a queue before there is slow work, or a graph database before there are relationship questions. Good system design is not memorising components. It is identifying a constraint, choosing the smallest change that removes it, and keeping enough evidence to know whether the decision worked.

Practical checklist

1. Define the user operation and the non functional target.
2. Measure where time, failures, and cost are actually going.
3. Introduce one component that directly removes that constraint.
4. Add monitoring and a recovery path before calling the change complete.

Related practical guides

• Shared data and scaling in a manufacturing SaaS
• PostgreSQL constraints and query design
• Durable jobs, queues, and Pub/Sub