Why we insist on infrastructure as code
Our products sit on Cloudflare, GitHub, Stripe, Supabase, Storyblok and Resend. The configuration of all of them lives in one repository, and our rule is that anything that can be code is code. Opening a dashboard is limited to the handful of steps that have no API at all.
This post covers how it is put together, and why.
Why we chose Pulumi
We split cloud accounts at roughly one to three domains each, so that a failure or a
mistaken click stays inside one product. The number of providers therefore grows as accounts
are added, and we picked Pulumi with TypeScript because it lets us walk the list of accounts
(config/accounts.ts) and build the providers from it.
// The idea behind providers/factory.ts
for (const account of cloudflareAccounts) {
providers[account.name] = new cloudflare.Provider(account.name, {
apiToken: process.env[account.tokenRef], // config holds the secret's name, never its value
});
}
A product only declares which account it belongs to (cfAccount: 'miyoshisoya') and gets
the matching provider injected. Adding an account is appending one element to an array.
Honestly, that difference alone is not decisive against Terraform (OpenTofu). What we have
appreciated in practice is that Pulumi encrypts the secrets in a stack’s configuration with
a passphrase and carries them for us. The size of the provider ecosystem turned out to be
fine too, because Pulumi bridges Terraform providers: Stripe uses the official
stripe/stripe, and Storyblok and Resend use community providers through that bridge.
pulumi package add terraform-provider stripe/stripe
Secrets are committed, encrypted
Secrets ended up handled like this.
- The real values are encrypted with SOPS and committed as
secrets/*.enc.yaml - Only two keys live in 1Password: the age private key and the Pulumi passphrase
config/holds the name of a secret (tokenRef), never a value
Keeping encrypted secrets in the repository means a change to a secret shows up as a pull request diff. Who replaced which token, and when, stays in the history, so every change goes through review before it lands.
Setting up a new machine takes the two keys from 1Password and a git clone. The keys are
read with op read, so unlocking with Touch ID is the only manual step, and passing files of
environment variables around went away with it.
make apply arukutomaru-prd
# bin/pulumi reads the two keys with op read, injects the tokens with sops exec-env,
# logs in to the R2 backend, and only then runs pulumi
Rotation is a single step in this shape: run sops updatekeys across the files and the
re-encryption is done. Keeping a state where every secret is listed and any of them can be
rotated at any time was the main goal of the design.
Pulumi mints the API tokens too
Exactly one Cloudflare API token per account is held by a human. That token authenticates
Pulumi, and every other token is minted from it as a cloudflare.ApiToken resource: one for
deploys, one for creating R2 buckets and D1 databases, one for the Secrets Store, one for Zero
Trust, and one per zone for DNS.
The permissions each token carries are declared in config/products.ts.
deployTokenPermissions: [
"Workers Scripts Write", // wrangler deploy
"Pages Write", // Pages deployments
"D1 Write", // D1 bindings and migrations
"Account Settings Read", // wrangler reads account info
"Secrets Store Write", // authorises the Worker's secrets_store binding
],
deployTokenZonePermissions: ["Workers Routes Write"],
So “what can this product’s deploy token do” is reviewable as a pull request diff, and widening a permission leaves its reason in the commit message. That is the difference from ticking one more checkbox in a dashboard.
The minted tokens live in a dedicated tokens stack, and the stacks that consume them read
them through a StackReference. That shape came out of a real problem: chaining a minted
ApiToken.value straight into a downstream provider leaves that value unknown at preview
time, so resources underneath show up as +- replace with an empty diff, and data sources
crash on empty responses. A StackReference holds an applied value, which is known at preview
time, and the false diffs go away.
One more thing worth knowing: the bridged Cloudflare provider does not diff changes to
permissionGroups. After adding a permission we recreate the token with
pulumi up --replace <urn> and push the new value back into the consuming stacks. That step is
written down in the docs.
What has no API, and stays manual
These four have no API at all, so they can only be done by hand in a console. Exactly those steps get a written procedure, so that anyone who follows it arrives at the same result.
- The Stripe account itself. The Terraform provider has no resource that creates one.
- Creating a GitHub App. GitHub exposes no API for it; it is a UI-only flow. Its private key and IDs are managed by infra afterwards.
- Turning on Cloudflare Email Routing. Cloudflare locks the feature toggle and the generated MX and DKIM records. Only the rules are ours to manage.
- Verifying an email forwarding address. Clicking a link in a confirmation email cannot be scripted.
What we care about is idempotency. Where the code holds the infrastructure, running it again converges on the same state; where only hands can do the work, the written procedure converges on the same state. Writing the manual steps down means that rebuilding the same environment, or handing it to someone else, produces the same thing.
What this bought us
Adding a product became an edit to a config file. The repository, the deploy token, DNS,
the Pages project, the GitHub Environment and its secrets all come from about twenty lines in
config/products.ts. This very site was created that way.
Permissions can be kept tight. The permissions a deploy token needs are declared per
product, so each token keeps exactly the scope it needs and any change to that scope is a
diff. This site has no Worker, so its token carries two permissions: Pages Write and
Account Settings Read.
We also think this puts us ahead of an organisation that already has infrastructure built by hand. Codifying an environment that exists means reconciling the real thing against the code one resource at a time, and that work is hard to see the end of. Build it as code from the start and the real thing always matches the code, so standing up a new environment leaves the old one exactly as its code describes.
If you would like to talk about infrastructure or a migration, please get in touch.
Nami Smart LLC
Fast, cost-efficient, high-quality app development. If you would like to talk about a project, please get in touch.