Key questions to ask in your apps testing phase

Most testing phases ask the wrong questions. Teams spend weeks checking whether buttons work, whether screens load quickly enough, whether the sign-up form submits correctly, and then they ship a product and discover that users do not understand what it does, cannot get through the first three steps, or abandon it before they experience anything worth coming back for. The technical questions were all answered. The product questions were never asked.

The testing phase should challenge your assumptions, not confirm what you already believe.

We see this pattern with apps across beauty, fitness, food, and hospitality. The testing phase becomes a quality assurance exercise, a search for bugs rather than a search for understanding. And the distinction matters enormously, because a bug-free product that confuses its users is still a failing product.

The questions below are the ones we return to on every pre-launch engagement. Some are about behaviour, some are about evidence, some are about timing. All of them are harder to answer than "did the payment flow work?" and all of them are more likely to determine whether your product survives its first three months.

Why the testing phase produces the wrong questions

Testing phases tend to be shaped by whoever is running them. Engineers test for what can break. Designers test for what looks right. Founders test for confirmation that they were correct. Each group finds what it is looking for, and the things nobody is looking for go unnoticed until real users arrive and make them visible.

There is also a structural problem. By the time a product reaches testing, a significant amount of money has already been spent. Budgets are tighter, timelines are compressed, and the team is emotionally invested in shipping. That combination creates pressure to confirm readiness rather than question it. Questions that might produce uncomfortable answers are quietly deprioritised in favour of questions that can be answered quickly and satisfactorily.

The result is a testing phase that validates the product the team built rather than the product the user needs. Screens get checked for consistency. Flows get walked through by people who already know where they lead. Nobody sits with a genuinely new user and watches what happens when they have no prior knowledge and no team member standing next to them to explain things.

Fixing this requires a deliberate shift in what you choose to test and who you choose to test with, before you start assembling your questions.

Are you testing assumptions or confirming them?

Every product is built on assumptions. The founder assumed users wanted a particular feature. The designer assumed a particular layout would feel intuitive. The copywriter assumed a particular phrase would land clearly. Assumptions are inevitable. The problem is treating untested assumptions as though they have already been confirmed.

A testing phase built around confirmation looks like this: you show your product to people who already understand the category, ask them whether the features make sense, receive broadly positive feedback, and call that validation. What you have actually done is find people who share your mental model and confirm that your mental model is internally consistent.

Real testing looks different. It puts the product in front of people who do not share your assumptions and watches what happens. It asks open questions rather than leading ones. It treats confusion as information rather than exception. And it is genuinely willing to find that something does not work, because finding that in testing is far cheaper than finding it after launch.

We asked a pre-launch founder in the football industry, who arrived with a detailed colour-coded spreadsheet mapping out competitor products, why users would choose an all-in-one product over specialised apps that did each job better. The founder had not tested that question. They had assumed the answer was obvious, and the deflation when we asked it was a signal that the assumption had never been properly examined.

What does a validated user need actually look like?

Founders frequently arrive with confidence that a user need exists because they have experienced it themselves. That is a reasonable starting point, not a finished argument. Personal experience proves that one person has a problem. It does not prove that enough people share it, that they want it solved in the way you are proposing, or that they would pay for the solution you have built.

Validated user need means confirmation gathered from people who are not the founder, not the founder's immediate circle, and not recruited specifically because they are likely to agree. Focus groups, surveys, and observational sessions with representative users are the tools here. The standard we apply before recommending a launch is that user need has been confirmed via people outside the founding team, through a structured process, and that the findings have been documented rather than summarised informally in a Slack message.

A validated user need is confirmed by strangers, not by people who already agree with you.

One thing worth noting about sample size: five users is often enough to surface the majority of usability problems, and Nielsen Norman Group's research puts that figure at around 85% of issues caught. But if a stakeholder challenges whether the sample is large enough, particularly when the findings suggest significant structural changes, our approach is to table those specific points and, if the concern persists, move from exploratory qualitative sessions into larger-scale quantitative surveys. More voices reduce the argument surface considerably.

Are you watching users or asking them?

There is a meaningful difference between watching someone use your product and asking them what they think of it. Both have value, but they produce different kinds of information, and using only one of them leaves a significant gap.

When you ask users what they think, you get their conscious opinion, shaped by what they believe you want to hear, what they can articulate in the moment, and what they feel confident saying out loud. That data has uses. But it does not show you where they paused, where they went the wrong way, or where they pressed something in confusion and then recovered without mentioning it.

Observational testing in practice

Observational sessions are specifically designed to catch behaviour that users would not report. The facilitator asks someone to complete a task, then watches without intervening. The signs we look for are pauses on a particular screen, incorrect route choices that the user has to back out of, and moments where the user reads something twice without proceeding. These are indicators of cognitive overload, and they appear reliably in observational testing in ways they simply do not appear in post-session interviews.

What users say versus what they do

The two methods work together. Observation tells you where the friction is. Interviews help you understand why the user interpreted something the way they did. Running only interviews and skipping observation is a common shortcut that produces confident but incomplete findings.

Record observational sessions and timestamp the moments where users hesitate or make incorrect choices. Reviewing the recordings after the session surfaces patterns that are easy to miss when you are focused on facilitating the session itself.

Does the onboarding flow work for someone who knows nothing?

The people who build an app know it better than any user ever will. They know what each screen is trying to do, why the steps are in the order they are, and what the language on each button is referring to. That knowledge makes them genuinely poor judges of whether the onboarding flow is clear to someone arriving with none of it.

The only way to test onboarding properly is to recruit users who genuinely have no prior exposure to the product and watch them move through it without assistance. The question is whether they can complete it alone, and whether they understand what they have done by the end of it.

We tested two versions of a multi-step flow on a health and wellbeing product. One version simply showed a progress bar. The other primed users upfront by telling them what to expect before they began, including roughly how long the process would take. Without priming, drop-off rates ran at around 80 to 85%, usually within the first three or four questions. After introducing that upfront expectation-setting, completion rates rose to approximately 95%. The progress bar alone was not the issue. Users were abandoning because they had no frame for what they had walked into.

Test your onboarding with users who have never seen the product before, and ask them to narrate what they think each step is asking them to do. Their narration tells you more than their completion rate alone.

Is the core value visible within the first sixty seconds?

A user who cannot understand what your product does within the first minute of using it is unlikely to persist long enough to discover it later. This is not an abstract concern. According to Think with Google, 21% of users will give up on an app if they do not understand it right away. The first sixty seconds are the product's primary opportunity to make its value legible.

Testing this requires a specific kind of session. You give a new user the app, you say nothing about what it does, and at the sixty-second mark you ask them to describe what they understand the product to offer. Their answer tells you whether your value proposition is landing without assistance.

What "visible" actually means

Visible does not mean stated. A product can have its value proposition written explicitly on the first screen and still fail this test, because the user did not process the text in the way the designer intended, or because the surrounding interface undermined the message. Visible means the user can reconstruct the core offer from their own experience of the first sixty seconds, not from reading your copy.

If users consistently describe something adjacent to your actual offer, or describe a feature rather than the benefit, the value proposition needs work before the product ships, not after.

Are drop-off points telling you something you are ignoring?

Drop-off data is one of the most consistently underused sources of information in a testing phase. Teams notice that users are abandoning at a particular step, note it down, and then move on without asking why. The number goes into a report. The reason behind the number does not.

Drop-off points are almost always communicating something specific. It might be that the step is asking for something the user is not ready to give, a permission request that arrives before the user trusts the product. It might be that the interface presents too many choices at once. It might be that the language on a particular screen is ambiguous. Each of these has a different fix, and you cannot choose the right one without understanding which applies.

Reading the pattern, not just the rate

The pattern of drop-off matters as much as the rate. If users are abandoning consistently at step three of seven, something about step three is the problem. If they are abandoning across a range of steps, the issue is more likely to be systemic, a flow that is too long, or an expectation that was not set at the start. Research from UXCam's analysis of 1,847 bailed sessions found that 58% of abandonment clustered around the experience of "no obvious value yet", which is a systemic issue rather than a step-specific one.

For every drop-off point you identify in testing, ask two questions before writing it up: what is the user being asked to do at that point, and what do they understand the purpose of that action to be? The gap between those two answers is usually where the problem lives.

Have you separated founder preference from user evidence?

Founder conviction is a genuine asset. It keeps projects moving through difficult moments and provides the focus that diffuse decision-making cannot. But the same conviction that drives a product forward can also prevent it from responding to evidence, particularly when the evidence conflicts with the founder's original vision.

We have seen this pattern on two separate projects, one in fitness and wellness, one in grassroots football, where founders with strong personal conviction in their product vision consistently set aside data coming out of research studies, focus groups, and surveys. The early warning signs were consistent: an escalating drive to perfect the product before launch, decisions being rolled back after sign-offs were given, and a prolonged decision-making process that kept absorbing budget. Both projects exhausted their budgets in design and build without ever reaching a fully live product.

The testing phase is the right moment to draw a clear line between what the founder believes and what the evidence shows. That means documenting both, separately, and being willing to discuss them as distinct things rather than assuming they point in the same direction.

A simple check

For each product decision currently in play, ask whether it is supported by user evidence, by founder preference, or by both. Where the answer is "founder preference only", that decision needs either testing or a documented rationale for proceeding without it. The gap between the two columns, founder preference and user evidence, is where vanity products are built.

Are you testing security and data handling, not just usability?

Security and data handling are frequently deferred to a post-launch phase, treated as infrastructure concerns rather than product concerns. That deferral carries real risk, both to users and to the product's reputation, and it is considerably cheaper to address these questions during testing than after a breach or a regulatory issue.

The questions to ask are specific. How is user data stored, and where does it live? What happens to data when a session ends? Who has access to it, under what conditions? Are permissions requested at a point in the journey where the user understands why they are being asked? According to Localytics, apps that request permissions in the first session see up to 60% lower opt-in rates compared to apps that wait until the user understands the value being offered. Timing a permission request well is a security design decision, not just a UX one.

On the performance coaching survey app we built, we used iOS's built-in libraries to generate QR codes entirely within the presenter's native app, so the codes were never stored externally. Once the presenter marked a survey as finished, no further responses could be recorded, making the survey time-limited and removing an open-ended vulnerability that a more standard implementation would have introduced.

That kind of decision gets made during design and build, but it needs to be verified during testing. Does the implementation actually behave as intended? Does the data handling match what users are told it does?

What should you not commit to publicly before testing is done?

The most common timing failure we see is taking too long, watching competitors enter the space while the product is being refined past the point of useful return. But a close second is committing publicly to dates and features before testing has confirmed the product is ready to deliver on those commitments.

We advised two clients, one on a gaming app for art enthusiasts and one on a grassroots football product, not to run paid social campaigns or announce fixed launch dates until after a soft launch. Both chose not to follow that advice. Both picked arbitrary dates, began posting about those dates publicly, and then encountered development changes that pushed the launch past what had been announced. Both then had to update their dates publicly, more than once, in ways that eroded the audience confidence they had been building.

The posts hinting at the product were not the problem. Committing to a specific public date before testing had confirmed the product was ready was the mistake.

What to hold back

The things worth keeping private until testing is complete include any announced launch date, specific feature promises tied to a date, and pricing or offer structures that would require walking back if the product changes. Generating interest before launch is sensible. Creating commitments you cannot yet keep is not.

How to structure the debrief so findings actually change decisions

Testing findings that change nothing are documentation. The debrief is the point where observational data and user feedback either shape what happens next or get filed away, and the difference between those two outcomes is almost entirely structural.

The debrief process we follow has a clear sequence. First, establish what was tested and what the research was trying to achieve, whether that was feature exploration, validation, or something else. Second, review who the participants were, because not every user is equally relevant to every feature, and weighting feedback appropriately requires understanding the profile of who gave it. Third, map the business value and likely user uptake of each feature or finding under discussion. Fourth, apply an effort versus impact assessment to determine what gets worked on next, what gets deferred, and what gets dropped entirely.

Findings that don't change decisions are just documentation filed away, not research that earned its place.

The sequence matters because it prevents the most common debrief failure, which is moving directly from "here is what users said" to "here is what we should build", without stopping to ask whether the finding is coming from the right users, whether it is worth the effort to address, or whether it serves the business at all. A finding that every participant mentioned is still worth dropping if fixing it would require six weeks of engineering work and affect 3% of the user base.

1. State what was tested and the research goals.
2. Review participant profiles and weight feedback accordingly.
3. Map the business value and potential uptake of each finding.
4. Apply effort versus impact to prioritise, defer, or drop.

That process turns a debrief from a summary meeting into a decision-making meeting, which is what it should always have been.

Conclusion

The testing phase is the last structured opportunity to find out whether the product you have built is the product your users actually need, before they make that judgment themselves in the market and you lose the chance to shape what they find.

The questions in this article are harder than "does the sign-up form work?" and they take longer to answer. But they are also the questions that determine whether the product survives its first three months or becomes another app that users open twice and delete. The technical testing still matters. Bugs are real, load times affect retention, and accessibility problems exclude users who should be included. But technical readiness without product readiness is a well-built thing that users walk past.

What we have seen on the projects where testing genuinely changed the outcome is that the team was willing to sit with uncomfortable findings rather than rationalise them away. The willingness to ask whether founder preference and user evidence are actually aligned, whether the onboarding genuinely works for someone with no prior knowledge, and whether the drop-off patterns are being read or just recorded, is what separates a testing phase that produces a better product from one that produces a launch date.

If you are heading into a testing phase and want a clearer view of which questions to prioritise and how to structure the findings so they actually feed into decisions, let's talk about your testing process.

Frequently Asked Questions