User research in the context of testing for launch is less about discovering the problem and more about validating the solution. At this stage the question shifts from what do people need to does what we have built actually meet that need in the way we intended. That is a different kind of research and it requires a different set of questions.
Three categories of pre launch research questions were introduced this week and I wanted to apply each of them directly to our project to make them meaningful rather than abstract.
Usability asks whether people can actually use the thing you have built. For Flare this would mean asking whether an intake officer can move through the consent and data capture process on their tablet without friction or confusion. It would mean asking whether the system confirmation is clear enough that the prisoner understands what has been paused on their behalf. Usability testing at this stage would surface any points in the service where the experience breaks down in practice even if it looked clean on paper.
Concept validation asks whether people understand what the thing is and believe it does what it claims to do. For Flare this is a particularly important question. The concept relies on trust. A prisoner opting in to the system is trusting that their obligations will actually be paused and that the system will not create new problems for them in the process. Concept validation research would test whether that trust is earnable and whether the service communicates its purpose clearly enough to earn it.
Mental model fit asks whether the way the service works matches the way users expect it to work. For Flare this means asking whether the intake process feels intuitive to the officer using it and whether the release summary makes sense to Dean when he is presented with it. If the system works in a way that conflicts with how people naturally think about their finances or their obligations, that mismatch will cause friction even if the underlying service is technically sound.
The session included an exercise asking us to define our own research questions at two levels. High level questions to frame the study and specific questions to tell you what to look for within it.
Applying this to Flare, my high level questions would be as follows. Does the system successfully prevent avoidable debt accumulation during a sentence? Does the intake process place an acceptable level of burden on prison staff? Does the prisoner leave with a clear and accurate understanding of their financial position?
The specific questions sitting underneath those would include things like. At what point in the intake process does the officer feel uncertain about what they are being asked to do? Does the prisoner understand what opting in means for their specific obligations? Is the release summary presented in a way that feels actionable rather than overwhelming? Does the system confirmation give the officer enough confidence that the process has worked?
Framing research at both levels is useful because the high level questions keep the study focused on what actually matters while the specific questions give you something concrete to look for during testing. Without both you either end up with findings that are too vague to act on or observations that answer questions nobody was asking.
<aside> 💡
This week was a useful reminder that research does not end when the building starts. It is easy to treat the discovery and define phases as the research phases and then move into development as if the questions have all been answered. They have not. Testing for launch is where you find out whether the answers you thought you had were actually right.
Applying the three research question categories to Flare was a genuinely useful exercise. It surfaced some questions I had not fully considered before, particularly around mental model fit. We have spent a lot of time thinking about whether the system works technically and whether the cost case stacks up. We have spent less time thinking about whether the experience of using it feels right to the people at either end of it, the intake officer and the prisoner. That is the kind of gap that testing is designed to catch and it is something worth addressing before anything goes live.
The exercise also reinforced something that has come up repeatedly throughout this module. Good design is not just about having a good idea. It is about being honest enough to keep testing whether the idea is working and humble enough to change it when the evidence says it is not.
</aside>