Architecture · data plane · creating a cluster

How a cluster is created.

There is no single-machine database in prty. One request produces a primary, a replica and an arbiter in three zones, all voting from the first second. This page follows that request from the API to the connection string.

The concept

The default cluster has three members, one per zone:

a data member that starts as primary: a VM, its persistent disk, PostgreSQL and the quorum sidecar;
a second data member, the replica, in another zone, with its own disk. Replication is by PostgreSQL streaming, not shared storage, because a persistent disk can only attach to a VM in its own zone;
an arbiter in the third zone: a small VM that votes and stores no data, so the cluster can tell “the primary is dead” from “my link to the primary is broken” (quorum and failover).

The other shape, a primary and two replicas (three copies of the data, no arbiter), is created by the same steps with a second data member in place of the arbiter; see the data plane overview.

The arbiter is the cheap member: it is why a three-way vote costs two copies of the data, not three.

What each step does

Allocate disks. One persistent disk per data member, each in its member’s zone, empty and formatted. Each disk is recorded in a resource registry the moment it exists, so a crash cannot leak it.
Provision members. One VM per member, in three different zones, from the database machine image (PostgreSQL, the sidecar, the agents; no internet needed). A data member gets its disk. Settings (size, version, the cluster’s passwords, the quorum token) arrive as boot parameters; on AWS the VM is given no cloud permissions at all.
Start the quorum. The sidecars find each other, form a Raft group, and commit the first cluster record: generation 1, who is primary, which replicas must hold every acknowledged commit. The primary initialises PostgreSQL; the replica copies it and starts streaming. A machine starts PostgreSQL only in the role the committed record gives it.
Wait until ready. The control plane polls until the cluster reports that the primary accepts writes and the other members have joined. It has a timeout.
Configure networking. The cluster’s private DNS name is pointed at the primary and stamped with generation 1, and the connection string is built. The string lists the name and every member, so a client can reconnect without DNS after a failover (quorum and failover). Access is from inside the private network only.

The credentials are generated per cluster at this point: an application role that is not a superuser (it cannot promote a replica or change the cluster’s role), a replication role, and the token the members use with each other. No role can log in over the network as the PostgreSQL superuser.

Failure modes that matter

What fails	What happens
A zone has no capacity for the VM type	The provisioning step retries; if it cannot succeed, the operation rolls back: VMs and disks created so far are deleted and the cluster is marked failed. The workflow does not fall back to putting two members in one zone.
The control plane restarts mid-create	It resumes. Steps that already finished are skipped; the interrupted one runs again, and creating a resource that is already in the registry finds it instead of creating a second one.
The members cannot form a quorum	“Wait until ready” times out; the operation rolls back. Until the record is committed, no machine starts PostgreSQL as primary.
The replica cannot catch up	The cluster is not declared ready: the create does not complete with only a primary.
The failure happens after the point of no return	Every operation has a point of no return, after which it is not undone but goes forward or parks as needs attention for a person (control plane).
The cloud API rate-limits or errors	Backoff and retry, bounded; a step that keeps failing ends the operation as above.

Problems this design solves

“We forgot to turn on high availability.” There is nothing to turn on. A one-member cluster cannot be requested.
Leaked machines and disks after a half-finished create: the registry plus ordered rollback means failure leaves nothing running that is billed.
A replica in the same failure domain as the primary. Zones are assigned by the workflow, not chosen by the caller, and a persistent disk cannot attach across zones.
Two primaries after a reboot. A restarted machine asks the quorum what it is; it never trusts what its own data directory looks like.
Connection strings that break on failover because they hold a machine address: the name and the member list survive it.
Everyone is a superuser. Applications get a non-superuser role.

Objectives

Objective	Target	Status
Request to a cluster ready for writes	≤ 8 min	Estimate Our end-to-end test waits up to 450 s, which is a timeout and not a measurement. To figure out: record the real time from every AWS run and use the median and the worst.
Time to add a replica to an existing cluster	depends on data size	Estimate New VM boot (minutes) plus a full copy at the disk’s throughput, so the time scales with the database. To figure out: time it at 10 GiB, 100 GiB and 1 TiB.
A failed create leaves billed resources behind	0	Design Every resource is registered at creation and undone in reverse order. Still to test: a create that fails at every step, in the AWS suite.
A cluster with its members in fewer than three zones	never	Design Placement is decided by the workflow, not the caller.

The cost of those three members, next to a cloud’s own managed PostgreSQL, is on the cloud costs page.