Discovery, User Research, User Testing

DfE Discovery

Df E Logo 800

Hive IT were successful in tendering for the opportunity to undertake this 'Discovery for Statistical Dissemination Options' on behalf of the Department for Education (DfE). Over 12 weeks we’ve worked with the DfE to explore the needs of users of the DfE official statistics, and to make recommendations for how they can further strengthen their statistics “offer”. This work was part of their wider package to review the coherence of DfE’s statistics.

the challenge

Hive IT were commissioned to carry out a 12 week research project to better understand the needs of users and producers of the Department for Education’s statistics, and to identify options for modernising the dissemination of products.

The aims of the project were to:

● Explore who uses DfE’s official statistics outputs, how they currently use and access those outputs and if they meet their needs

● Explore how to best make those outputs available to users, learning from industry best practice ● Explore how different dissemination routes can free up statistics producers’ time and help them tell clear and coherent stories with the data

● Explore how they can reduce the need for ad hoc data requests by publishing more flexible data outputs

the result

Over the course of the project we worked with the DfE to identify who the users of the official statistics were and completed detailed user research and testing both within and outside of the DfE. We also worked hard to encourage engagement in the project across the organisation, working hand in hand with the DfE Product Owner. As a final deliverable we produced a comprehensive report detailing our findings which included clear recommendations for future steps and some smaller more incremental changes that could be implemented within shorter timescales.

“Over the last 12 weeks Hive IT have been working with statisticians in the DfE to take a fundamental look at the needs of users... The work has also given us some helpful practical next steps to take. It has been good to hear about how valuable many users find DfE statistics and data; much of what we already do meets the needs of a range of users. But it is clear we have more to do to modernise and to more fully meet users’ needs.”

Neil McIvor
DfE Chief Data Officer and Chief Statistician
Df E Discovery Hub
1.

approach

The project was run using the GDS (Government Digital Service) agile methodology for Discovery projects . As part of the project we set up ‘Project Hubs’ at both our Hive IT offices and the Sheffield and Darlington DfE offices to encourage visibility and engagement from other DfE departments and teams. These were regularly updated with information as we moved through the project. We worked in weekly sprints and held Show and Tells which we filmed and shared, again in an effort to foster engagement and promote visibility across the organisation. We worked hard to foster a ‘One Team’ ethos with the DfE, making sure that we encouraged an open, honest and collaborative working style through our defined Ways of Working.

Research4
2.

the research

The Discovery Project was broken down into 4 phases and ran from the 19th March - 12th June.

The phases of the project were as follows:

  • Phase 1 Internal research with the DfE teams producing statistics
    • This phase concentrated on bringing Hive IT up to speed with the work that the Data Insight and Statistics Division complete. Holding workshops in Sheffield and visiting Darlington to complete contextual research. 
  • Phase 2 Internal and external research with statistics users
    • This was an intense user research phase, focusing on exploring how the statistics are used across the organisation and by external organisations and individuals. Over this phase we conducted face to face interviews, contextual research and digital interviews.
  • Phase 3 User Testing and best practice for statistics production and dissemination
    • User Testing in Phase 3 focused on asking users to engage with systems that had been identified as potentially viable options for improving the dissemination of statistics. Analysis of best practise focused around technical reviews of existing tools and systems that are currently available within the marketplace, as well as forming part of the user testing.
  • Phase 4 Discovery wrap up and Alpha phase definition
    • Phase 4 concentrated on consolidating all the research and forming our recommendations supported by that research. Culminating in the final report that was delivered to the DfE along with all the supporting research.

Dfe Findings
3.

high level findings

Our key findings can be summarised as:

  • Many users are very appreciative of, and dependent on, the statistics produced by the DfE. For external expert consumers, in most cases their livelihoods and ability to do their jobs depend on it.
  • For internal users, even when skilled, their role means they feel the need to check their interpretation of DfE-published statistics and would like to be able to better self-serve. They want to be less of a burden and less reliant, and not face the delays this results in.
  • For many producers of statistics within the DfE there is a commitment to modernisation. Tied to this, there is a lot of frustration with the apparent inflexibility of GOV.UK as a platform for disseminating statistics, and with the time spent on routine tasks such as QA and Suppression.
  • All identified user types have a desire to be able to customise the data according to their requirements, they want consistency in data, ease of use, the ability to view historical data alongside current data and to be made aware of changes to datasets or when new publications are released more effectively.

Dfe Recommendations
4.

our advice

At a high level, with the knowledge that the Discovery phase provided us, our recommendations to the DfE were to:

  • Undertake an Alpha project focusing on internal process changes. This explores developing the existing work done on automating routine analysis and making the processes more easily reusable (known internally as the the Reproducible Analytical Pipeline or RAP) and expands its use throughout the Department
  • Undertake a separate second Alpha project focusing on the dissemination of data which provides a new platform for access to, and consumption of, statistics. This should make the statistics easier to find, access, navigate and understand.
  • Implement a number of smaller changes which are external to the Alpha projects but which will have significant impact

Df E Team
5.

retrospective

Throughout the project we completed weekly retrospectives in an attempt to continually refine and improve our processes and ways of working - as well as identifying issues that we could work to resolve. At the end of the project, we took the time to review the overall project with a final ‘Project Retrospective’.

Whilst there were some key things that we should work to adapt in any future project of this type, the overwhelming finding of this retrospective was that it was an enjoyable project made so by the drive, investment and belief of the whole team.

Our ways of working together fostered a ‘one team’ atmosphere where everyone felt valued, heard and able to talk freely about problems we were encountering. If there was one thing that we should try to replicate on any future projects of this type, it would be to model the same ‘team’ investment.

Dfe Product
6.

the finished product

We’ve talked at a high level about our research, findings and recommendations here, but we produced a more detailed report for the DfE which can be viewed here:

https://tinyurl.com/y85hpv3q

get in touch

related case studies