Scope of the pilot

The pilot had two main aims:

• to test the app with patients, so that findings could be fed into further development to improve the user experience
• to look at the impact it had on practice staff, so that we could find out what practices really needed to know before the wider rollout stage and ensure they were able to fully realise the benefits of the app

34 practices invited patients to take part in the pilot by text message or email. Patients had to complete a short survey to determine their suitability for the pilot. Patients were then emailed a link to download a test version of the app.

Our aim was to test the app with at least 2,000 users. We ended the year with more than 3,000 users from 34 GP practices across England.

Practices were recruited to be representative of practices across the country. They included rural and urban, small and large multi-location practices. They were a mixture of practices that already had a high proportion of patients that had GP online services, and others that were below average. They were spread across many different regions of England.  They were recruited to be part of the pilot either by the Empower the Person team, or directly by their CCG.

Research methods

A user researcher working on the NHS App project conducted user research through:

• phone calls

• semi-structured interviews

• agile software development methods, such as A/B testing

• online surveys
• in-app surveys
• analytics

Research participants

We conducted research with:

• operational staff

• clinical staff at practices
• patients that had the NHS App
• patients that didn't have the NHS App

The practices involved in the pilot used either TPP or EMIS systems, which represented nearly 95% of the GP system suppliers market as of December 2018. Testing will continue with other primary care GP IT systems suppliers that are currently being integrated into the NHS App.

How we used the findings

We used the feedback from the research participants and research methods to inform our Agile development pipeline by providing continuous updates and improvements to the NHS App throughout the pilot phase. Most issues identified in this report have already been resolved, or are currently being addressed by the NHS App team. We continually assess feedback as we receive it and make changes to the app as often as every two weeks.

This section contains some figures showing how the pilot developed to involve thousands of people of different ages who interacted with the app. By 21 December 2018:

• we received 14,175 completed surveys from potential users
• we sent out 12,851 email invitations to download a test version of the app
• 3,192 users registered on the app by 21 December 2018
• there had been 337 appointment bookings through the app
• there had been 106 cancellations of appointments through the app
• 662 repeat prescriptions had been ordered through the app

We achieved a nearly equal spread of users connected to practices that use TPP or EMIS:

However, more Apple than Android users registered for the app:

There was a good spread of age ranges using the app: