Not-so-secret learnings about kinda-secret projects (aka doing strategic user research)

I like to work in the open. Being transparent with users, stakeholders and colleagues about what user research is, how we do it, and what we’ve learned is important, because building collective understanding and buy-in leads to the better design of user experiences, and ultimately the solution of even knottier problems.

But working in the open can be difficult when assigned to strategic internal projects. Like what I’m doing now at Citizens Advice, which is to support the procurement of a new phones platform for thousands of staff and volunteers, ensuring it meets the diverse needs of those who will use it.

Or, what I did for a while at NHS Digital, which was to conduct user research to support the design of a new operating model. This sought to transform the way leadership and team structures worked collaboratively and across disciplines to deliver whole services to users, as opposed to the more traditional approach of being discipline-centered and product-focused.

There’s several, maybe obvious, reasons why both of these projects have to be, in some way, kinda-secret. A procurement process must follow strict commercial and legal procedures. Operating models are deeply sensitive areas of work, not least because they’re about strategic (re-)positioning, but also because they can directly impact individuals working in that organisation.

However, I did have some musings about doing strategic, sometimes quite nebulous, user research that I wanted to share.

Sensitivity is not an excuse to avoid user research

Working in the public and third sectors, it feels like sensitive projects are sometimes considered too complex, or too risky, for getting users involved. But it is possible. Throughout Citizens Advice’s procurement process, I built in opportunities for users to review potential solutions, gathering feedback and questions that I could incorporate into formal evaluations. Users just signed an NDA, in the same way that members of the team did. This also ensured user needs were represented and prioritised right from the very beginning, an important component of making user research more equitable.

I also found that getting users involved on strategic projects, early on in decision-making, could build trust and buy-in for the work ahead. Anecdotally, the users involved in Citizens Advice’s procurement process appeared enthusiastic and reassured that they could contribute to, and influence, such a big strategic decision. And at NHS Digital, with understandable concern from union reps about how operating model changes could impact staff, I feel that showing those reps that user research was conducted actually alleviated some of their concerns, because it tangibly demonstrated that the project was taking seriously the needs and perspectives of those affected.

User research can work in waterfall projects

A user researcher’s natural home is with other user-centered design disciplines, working in iterative cycles on agile projects. Procurement is quite a strict waterfall process, with many steps needing to happen in a particular way and in a certain order. I initially questioned how compatible this would be with user research. However, reframing these steps as ‘phases’ of procurement, I could see more clearly where research could fit.

For example, a discovery into user needs was clearly something that could influence the technical requirements given to suppliers in the initial tendering phase. When suppliers returned their bids, they could be assessed based on how well they met user needs.

And as submissions were shortlisted, and procurement moved into the competitive dialogue phase — where suppliers discuss their proposals in detail, answering questions and giving demonstrations where needed — this was an opportune place for users to be involved. I invited users to review the same demos that the project team had received, translating that feedback into questions to be asked during dialogue meetings.

I also worked closely with an accessibility consultancy to audit shortlisted submissions, supplementing their findings with direct user feedback around accessibility. It was powerful to be able to share technical comments about how a supplier used colour to convey meaning, for instance, and backing that up with quotes from a colourblind participant. This helped suppliers better empathise with Citizens Advice’s users, understand why this was so important for the project team evaluating the platforms, and ultimately led them to submitting final proposals that showed greater commitment to delivering excellent accessibility.

Get creative with methods

I found myself thinking back to my innovation days at Action for Children when working on these projects. Because, like innovation, you have to be creative when trying to understand how strategic change will affect users. How do you actually visualise and test organisational change? How do you test the usability of a platform that doesn’t exist yet?

At NHS Digital, instead of getting users to interact with prototypes during the Alpha phase, I roleplayed with real teams, trying to solve real problems, to understand if the hypotheses around new team structures, roles and responsibilities made sense. This reminded me of smoke testing fundraising products on social media, to see if users would engage with them — only to find that the product didn’t exist yet. There are ways to simulate real behaviour, even if you’re at the stage of a project where insights are more traditionally based on attitudes.

And at Citizens Advice, in lieu of being able to get users to directly usability test different platforms, I worked with the team to identify key user tasks that suppliers were asked to demonstrate visually. Users watched those demos and fed back on things they liked or didn’t like about how tasks were completed, reflecting on their own experience of completing these tasks on the existing platform. They also commented on things that surprised them, like when they expected a task to follow a certain journey but didn’t. This reminded me of lo-fi usability testing, when users have to describe what they think might happen next, even when they’re not interacting with a fully fledged prototype — an important way to learn about mental models even when you’re in the very early stages of a project.

I hope this shows that even in the most complex, strategic, fuzzy projects, user research can not only be done, but can meaningfully contribute to better strategic outcomes. I’ve learned how to adapt traditional research methods to more strategic work, too. And, of course, this blog has reinforced my belief that even a kinda-secret project has not-so-secret learnings that can be shared.

With thanks to Aneta Perehinets for proofreading.