What should I do if a new app update breaks something?

An app update breaks something live. Users start reporting errors. The support inbox fills up. Someone sends a Slack message with three question marks. This is not a hypothetical, it happens to teams shipping regularly, and how you respond in the first thirty minutes shapes how much damage gets done. The technical failure is one thing. The trust damage is another, and it takes longer to repair.

A broken update is rarely just a technical failure. It is a trust event, and trust takes longer to fix than code.

We have seen this play out repeatedly. On an auction game we worked on, engagement started dropping and the instinct was to add features, to give users something new to distract from whatever felt off. The wrong call. The core user journey had quietly degraded, and layering new functionality on top of a broken experience made things worse, not better. Retention eventually recovered, the figures came back to somewhere in the mid-sixties to low seventies, but not immediately after the fix went in, because trust that eroded over weeks does not return overnight.

That gap between fixing the problem and recovering the relationship with users is the part teams least expect. The code goes out, the bug is patched, and then people wait for numbers to move and they do not, not yet. Understanding why that happens, and what to do from the moment something breaks, is what this article covers. There is a sequence to get right, and skipping steps in the middle costs you more time than following them would have.

Stop the bleeding: your immediate priorities when an update breaks

The first priority is containment. Before anyone writes a ticket, opens a debugger, or convenes a call, someone needs to answer one question: how many users are affected right now? If you have a staged rollout in place, you can pause it before the broken update reaches everyone. If it has already gone to 100% of users, the maths changes and speed becomes the only lever you have.

Pause what you can, fast

Most app stores and deployment pipelines offer some version of a halt. Use it. A broken update still reaching new users is a problem you are actively making larger. Pull the handbrake on distribution while you assess the scope. This buys time without committing you to a full rollback decision, which is a bigger call and needs more information before you make it.

Assign a single decision-maker

Incidents where everyone is in charge end up with no one moving quickly enough. Designate one person to own the response from this moment forward. That person does not need to fix the bug, they need to coordinate, communicate, and make the calls that keep the team from pulling in different directions. Everyone else feeds information to them rather than acting independently.

Get eyes on your monitoring dashboards immediately. Crash rates, session drops, error logs, these tell you what is breaking and how often, which shapes every decision that follows. If your monitoring cannot answer those questions within five minutes of an incident starting, that is a separate problem worth addressing before your next release.

How to assess what actually broke and how badly

Once containment is underway, you need a clear picture of the damage. Not a rough sense of it, an actual breakdown of what is failing, for whom, and with what consequence. Without this, every subsequent decision is a guess dressed up as a choice.

Start by separating crash-level failures from degraded experiences. A crash that prevents login is categorically different from a slow animation or a missing thumbnail. Both deserve attention, but they do not deserve the same response time or the same level of alarm. Treat them accordingly.

Map affected flows, not just error codes

Error codes tell you where the system failed. User flows tell you what that failure cost. If checkout is broken in a skincare app during a sale period, that is revenue leaving in real time. If a preferences screen throws an error that no one was using this week, that can wait. Translating technical failure into user impact is the step teams skip when they are moving fast, and it is the one that helps you prioritise correctly.

Check whether the problem is universal or device-specific, OS-specific, or limited to a user segment. Look at when it started relative to the update push. Cross-reference error logs with user reports, because users often describe a symptom three steps downstream from the actual failure point. Building that picture takes twenty minutes and saves you from patching the wrong thing.

When to roll back and when to push a fix forward

This is the decision teams find hardest, partly because both options carry risk and partly because the pressure to do something visible is high. Rolling back feels safe. It often is not. Pushing a fix forward feels slower. Sometimes it is the right call.

The honest comparison looks something like this:

The presence of a data migration in the update is the most common reason teams cannot roll back cleanly. If the update changed the database schema, reverting the code without reverting the data leaves you with a previous version of the app trying to read a structure it does not understand. That is a second incident, and it tends to be worse than the first.

Rolling back feels safe and often is not. A data migration in the same release changes everything.

If a rollback is genuinely clean, and the previous version was stable, and you can execute it within the hour, then roll back. Give yourself room to fix the problem properly rather than rushing a patch out under pressure. A patch written in forty minutes of panic rarely solves things cleanly and often introduces a second failure.

How to communicate with users during an outage

Say something before users have to ask. The gap between a problem starting and an acknowledgement appearing is where trust erodes fastest. Users are unreasonable about silence, because silence reads as indifference.

What to say and where

Your first message does not need to contain a solution. It needs to contain three things: acknowledgement that something is wrong, confirmation that you know about it, and a rough indication of when you will next update. That is enough. A status page update, a pinned post on social, an in-app banner where technically feasible, push the acknowledgement through every channel users are likely to check.

Avoid vague language. "We are aware of some issues" tells a user nothing about whether their specific problem is included. "Checkout is currently unavailable for some users and we are working on a fix" is specific, honest, and actionable. Users can decide whether to wait or come back later. That choice is something you can give them.

Keep updating, even when nothing has changed

The instinct is to wait until there is news. Resist it. A message that says "still working on this, no update yet, we will post again in 30 minutes" costs almost nothing to send and does meaningful work. Silence between updates reads as abandonment. Regular cadence reads as competence. The content of the update matters less than the fact of it arriving when you said it would.

Set a fixed update cadence at the start of the incident, every 30 minutes, every hour, and stick to it even when there is nothing new to report. Consistency signals that someone is still on it.

Why broken updates rarely come from nowhere

A failed update is usually not a random event. It is the visible end of a process that started weeks or months earlier, in decisions that seemed low-risk at the time. The actual breaking point is often small, a dependency version change, a config value, an assumption about state, but the conditions that made it catastrophic were already in place.

One pattern we see often is product coherence failures, where significant effort goes into one part of a product while another part is built on assumptions. On a dating app project we worked on, we invested heavily in onboarding verification, the logic being that trust was the core value of the product. But the messaging feature was built generically, without the same level of discovery, and it allowed exactly the behaviour the onboarding was designed to prevent: fake, automated messages. The two parts of the product pulled in opposite directions because discovery had not touched both of them.

The same logic applies to updates. Teams invest in testing the new functionality and undertest the interactions between new and old code. The break happens in the seam, not the feature. Knowing that changes where you look during QA, at what was built and at everything it touches.

Before releasing any update that touches shared infrastructure or data models, map every other feature that reads from or writes to the same layer. Test those too, not just the new code.

Technical debt and the fragility it creates over time

Technical debt is the accumulated weight of decisions deferred, patches left in place, and dependencies not upgraded, and it makes every subsequent change more likely to break something it was meant to leave alone. Teams working in a high-debt codebase spend a disproportionate amount of time managing the consequences of earlier shortcuts rather than building new things.

We worked on a communications product where trust and security were central to the value proposition. The client repeatedly declined to allocate time to address the technical debt building up in the codebase. That debt stayed unresolved until the product became fragile enough that its reliability was visibly compromised, which, for a product built on trust, was a serious problem. Only at that point did the client agree to stop new feature work and spend the time reworking core mechanisms, improving core flows, and stabilising the product. The cost of that rework was significantly higher than it would have been if the debt had been addressed incrementally.

According to CodeScene, 2022, between 23% and 42% of the average organisation's development time is consumed by technical debt. That is time not going into features, testing, or stability work. The teams shipping the most fragile updates are usually the ones whose debt has quietly reached the point where every change is a risk.

Treat technical debt as a standing item in sprint planning, not something to address when a quiet sprint appears. Quiet sprints do not appear on their own.

What good monitoring and feedback loops would have caught earlier

Users who hit a broken experience after an update frequently do not report it. They close the app, come back later if they remember, and form a quiet opinion about whether the product is reliable. The absence of support tickets is not a sign that things are working. On mobile especially, silent abandonment is the dominant response to a poor experience.

Between funding rounds on the auction game project, we made the decision to implement proper user retention tracking and to start proactively asking users how things were going. The previous assumption had been that investor confidence was a reasonable proxy for product health. It was not. Real retention data looked different from what confidence suggested, and the gap between assumed and actual health was significant. Getting that data earlier would have changed decisions that were made later and at higher cost.

Good monitoring ahead of a release gives you a baseline. Crash rate, session length, drop-off points, error frequency, all of these need a pre-update benchmark so that when something changes after the release, you know within hours rather than days. A meaningful spike in crash rate at 2am on a Tuesday is only visible as a spike if you know what normal looks like.

What to track, specifically

• Crash rate by OS version and device type
• Session drop-offs at key flow steps, particularly checkout, login, and onboarding
• Error frequency in API calls, especially on first launch after an update
• User retention at day 1, day 7, and day 30, tracked against pre-update cohorts

These are the minimum you need to detect a problem before users start leaving and not coming back.

How to conduct a proper post-mortem without it becoming a blame exercise

The post-mortem matters as much as the fix. A team that patches the immediate problem but does not understand how it happened will hit a version of the same failure again. The goal of the post-mortem is to find the systemic cause, the conditions that made this possible, not the individual decision that triggered it.

This distinction is important because blame-focused post-mortems teach people to hide information. A developer who fears being identified as the person who introduced the breaking change will not surface relevant context freely. That context is usually the most useful part of the whole exercise. The question to ask is not "who did this" but "what conditions made this possible and what would have caught it earlier."

A structure that works

1. Timeline: reconstruct what happened, when, and in what order, without interpretation
2. Detection: when did the team first know something was wrong, and how?
3. Response: what was decided, what was the effect, and what slowed things down?
4. Root cause: what systemic condition allowed this failure to reach users?
5. Prevention: what change to process, tooling, or practice would address that condition?

Write this document and share it across the team. Not as a record of what went wrong, but as a record of what the team learned and what it is changing. Teams that run post-mortems this way get better at shipping. Teams that skip them ship the same failure twice.

Involve the right people

Include everyone who was involved in the incident response, not just engineering. If a product decision shaped the conditions, a deadline that compressed testing, a scope change that arrived late, the people who made that decision need to be in the room. Post-mortems that are purely technical miss the organisational causes, which are often the most significant ones.

What needs to change before the next release

The post-mortem produces findings. Those findings need to become process changes before the next update goes out, not aspirations for some future sprint. If the answer is "we need better test coverage on the checkout flow", that work belongs in the next sprint. If it is "we need a staging environment that more closely reflects production", that is a planning conversation that happens this week.

The most common gap we see after an incident is teams generating good post-mortem output and then returning to their normal release cadence without implementing any of it. The next release goes out under the same conditions as the one that broke. This is the pressure of shipping taking priority over the structural work that would make shipping safer. But the cost accumulates, and the next incident is usually worse than the last.

Minimum viable process improvements

• A rollback plan documented before every release, not written during an incident
• A staging environment tested against production data structures before deployment
• Monitoring baselines captured pre-release so post-release changes are visible immediately
• A staged rollout for any update touching core flows, with automatic pause triggers if crash rate spikes

None of these are complex to implement. They are slow to implement without clear ownership, which is why assigning the post-mortem actions to named individuals with deadlines is the step that determines whether anything actually changes. Shared responsibility for process improvement tends to mean no one owns it, and then the next release goes out unchanged.

Conclusion

A broken update is a system failure, and the system includes the code, the process, the communication, and the decisions that accumulated in the months before the release. Fixing the immediate problem gets the app working again. Understanding the conditions that produced it is what makes the next release safer.

The sequence matters: contain the damage quickly, assess what broke and how badly, make the rollback-or-fix decision on evidence rather than pressure, communicate with users before they have to ask, and then use the post-mortem to close the actual gap rather than just the visible one. Skipping steps in that sequence is where most of the avoidable cost happens.

What we observed on the auction game project, retention recovering but slowly, trust rebuilt over weeks rather than days, reflects a wider pattern. Users return when the product is reliable again, but the speed of that return depends on how the incident was handled as much as how quickly the fix went out. A team that communicated clearly, moved decisively, and demonstrated that it understood what happened recovers faster than one that went quiet and patched things silently.

The work after the incident is as important as the response during it. If you are working through a broken release or trying to build a process that catches these failures earlier, let's talk about your release process.

Frequently Asked Questions