I have watched small engineering teams burn six months building a deploy pipeline before the first feature shipped on top of it. I have also watched teams skip the pipeline entirely and let a senior engineer push from a laptop on a Friday afternoon. Both groups learned lessons the hard way. The middle path, where the deploy machinery gets built and operated as a real production system, is the one most teams actually live in. The interesting question is whether your team should be the one running it.
What runs between git push and “live for users”
A deploy does not move your source code to a server. It moves the output of a build of your source code. The build step compiles, bundles, resolves dependencies, runs tests, and emits an artefact (a single frozen binary, container image, or packaged bundle that represents one exact version of your app). The artefact is what gets shipped.
What that build looks like depends on the stack. A Go service compiles to a static binary. A JavaScript front end gets bundled. A Python app gets a frozen dependency tree. In most modern setups, the output is a container image (a packaged snapshot of the application plus everything it needs to run, ready for any Linux host that supports the container runtime), and the artefact sits in a registry until the deploy step pulls it.
Between the build and the deploy sits the long middle: schema changes, readiness gates, traffic shifts, and a rollback path. Each step is mechanical. Each step is also where most incidents start.
Schema changes are the dangerous step
Code is replaceable. Data is not. A bad code version is one release away from being gone. A migration that corrupts a column or drops rows can be unrecoverable without a backup older than you want to admit.
The safe pattern is the backwards-compatible migration. Add the new column first, ship code that handles both old and new shapes, then drop the old column in a later release. Each step is safe in isolation. It is also more steps than a single-shot migration, which is why teams cut it. Most migration incidents I have seen came from teams that took the shortcut.
The other common failure mode is the hand-run migration. Someone runs the migration directly against the database from their laptop because the deploy system is “down for maintenance.” The team forgets that happened. The next deployer assumes the schema is at the version they expected, and the application crashes on startup. A platform that logs every migration with a timestamp prevents this. A bespoke pipeline that does not log is asking for the failure.
Readiness gates are the gate
Once the new version starts, the platform does not trust it. It polls an endpoint (usually a route that returns “OK”) and waits for green before sending real traffic. Two flavors exist. Readiness asks “are you ready for traffic yet?” Liveness asks “are you still working, or should I restart you?” They look similar but matter when a service is alive but not ready, like an instance that is still warming a cache.
Without a working readiness probe, a deployer has two choices. Ship traffic immediately and hope the new version is healthy by the time users hit it, or wait an arbitrary number of minutes and hope the wait was long enough. Both are guesswork. A platform that polls the endpoint and shifts traffic on green is doing what your sleep timer was approximating.
Rolling updates replace the plane mid-flight
The classic pattern is a rolling update. Four copies of your service are running. The deploy starts one copy of the new version, waits for its readiness probe to pass, shifts a slice of traffic to it, then retires one copy of the old version. It repeats until only the new version remains. At every step, enough healthy copies are alive to serve real traffic, so users do not notice the swap.
Variations exist. A blue-green deploy runs the full new version beside the old one and flips traffic atomically once the new version is verified. A canary release sends one percent of traffic to the new version first and watches the error rate before widening. The pattern you pick matters less than whether you have one at all.
Hand-rolled traffic shifting is where I have seen the worst engineering tax. The logic is straightforward in outline, but every edge case (a step fails halfway, a probe times out, the load balancer drops a route) needs its own handling. Teams that build this themselves spend months on it and maintain it forever.
Rollbacks are the escape hatch
Sometimes the new version passes every probe and still breaks something real. An error rate climbs. A page renders blank. The fastest fix is almost never a new patch. It is a rollback, redeploying the previous artefact you already know works.
A rollback is only fast if the old artefact is still stored, versioned, and runnable in seconds. If your team has to rebuild from an old commit to roll back, you are gambling at the worst possible moment. Most platforms keep the last several releases ready to redeploy with one command. Bespoke pipelines that lose history on every release cannot promise the same.
When the platform is the wrong answer
The case for handing deploy to a managed platform is strong. It is not universal. Three situations where owning the pipeline is the cleaner call:
- Regulated workloads. Finance, healthcare, government, and similar industries often have requirements a managed platform cannot satisfy out of the box. Data residency rules. Audit trails that need to extend into the build environment. Security controls that touch the build step itself. In these cases, the cost of operating your own pipeline is real but it is the cost of operating in that industry.
- Deploy is your product. If your company sells CI/CD, release orchestration, or internal developer platforms, your pipeline is the product, not overhead. The same applies to platform engineering teams whose explicit job is to build and own the deploy layer for dozens of internal teams.
- Workloads outside the standard envelope. GPU clusters with strict driver versions. Real-time systems where a one-second deploy is too slow. Hybrid on-premise and cloud setups. Hardware-in-the-loop test rigs. Managed platforms are optimized for stateless web services and standard container workloads. If you sit outside that envelope, shoehorning into a managed platform often produces more friction than narrow purpose-built tooling.
- You already have a platform team. Some larger organizations have an internal platform team whose charter is to operate the deploy layer for dozens of product teams. The cost is real but it is shared across many feature teams, and the per-team tax is small.
The common thread is specificity. Teams that are correct to own their pipeline can usually state clearly why a platform does not fit. “We have always done it this way” is not the reason. “Our compliance requirements mandate X” or “we sell deploy tooling” is.
Trade-offs
Handing deploy to a platform is not free in time. You still need to define build commands, readiness probes, environment variables, secrets management, and migration ordering. The platform handles the mechanics, but the application-specific configuration is yours.
Vendor lock-in is real. Each platform has its own deploy manifest format, its own way of defining probes, its own rollback semantics. Moving from one to another is a project. The escape hatch is keeping your artefact build generic (a Dockerfile plus a binary plus a manifest the platform can read) so the platform-specific surface stays thin.
Cost moves from “engineer time maintaining pipelines” to “platform bill per release.” For a team deploying ten times a day, the platform bill is usually smaller than the engineer time. For a team deploying twice a month, the math may flip. Run your numbers.
For teams running regulated workloads or selling deploy as a product, the platform answer is the wrong answer. The trade-off is permanent engineering cost versus permanent vendor cost. Each team picks the one that matches their constraints.
What I would tell past me
If I could send a message back to the engineer I was when I first ran a production release, I would say three things.
- The build is the cheap place to catch bugs. Every minute you invest in tests and lint runs at build time saves an hour at 2 a.m. during an incident. Failing the build is a feature.
- A rollback you have not tested is not a rollback. Schedule a monthly drill where you roll back the staging environment. The first time you find out your rollback is broken should not be during a real incident.
- You probably should not own this. Unless your business is deploy, your engineers’ hours are worth more on product work than on pipeline maintenance. Run the numbers honestly. Include on-call time.
A decade ago, owning your pipeline was unavoidable. Today it is a choice, and for most teams it is the wrong one. Know how the chain works, because on-call at 2 a.m. demands it. But knowing how it works is not a reason to own it.