Goals of Refinement (Field) Testing


A   summary of our   broad goals   for refinement (field) testing is to determine:

  • What's really happening : understand the context on the ground in which the app will be used
    • Are questions in an order that doesn't match the users' current workflow?
    • Are inputs required that will realistically not be available?
  • How the system will be used:   observe app interaction during and app usage after training to pinpoint potential barriers to app usage
    • Is the app taking too long to use?
    • Is a question better as open text than multiple choice?
  • What the users want:   get end user feedback on what would work best and be most useful for them
    • Should we add something that the users want?
    • Should we remove something that the users find irrelevant?


A note on duration:   In order for us to gather meaningful feedback, refinement (field) testing needs to run for a reasonable amount of time. Ideally, the duration would be around 2 months, but timelines not permitting,   1 month   is what we would aim for. Certainly anything less than 2-3 weeks will limit our takeaways and consequently how user-friendly and robust the next version of the app would be. Anything   less than 2 weeks   will not give us the level of feedback we are looking for, but may allow for some very basic testing.


The takeaways from a comprehensive process are related to workflow as well as app design. Specifically, the following things are important for us to do.


1. Tablet:   Observe how the tablet is handled by the users in order to:

  • confirm the   appropriateness   of this device for the project,
  • guage the level of   comfort   the users have with the tablet, and
  • inform preparations and schedules for   future trainings .


2. Printing:   If the printing functionality is ready and printers have been procured by the Karnataka launch, observe how the printing mechanism fits into the current workflow of the nurse and doctor, including interaction with the patient, to determine:

  • how the   doctors feel   about the printout, and
  • whether the printout will be in the   patient's possession   at any point.

These would be to confirm, or iterate on changes to ensure, that:

  • doctors do not have any issues with the   introduction of the printout ,
  • the printout will not be   lost or cause confusion ,
  • the printout will   effectively convey information   critical to the study between the nurse and the doctor.


3. Workflow:   Observe how the app fits into the daily workflow of the clinics, specifically of the nurses and of the doctors. This would allow us to:

  • confirm   who   should and would be using the tablets in a real setting,
  • know where tablets will be   stored , how they will be   charged , etc.,
  • iterate on app design so that it is   least disruptive   for the users, and not take too much time, or even save time (ex: adding, removing and reordering of questions), and
  • iterate on app design so that the   system's required inputs   are reasonable to expect at the CHCs (ex: making questions required vs not required, adjusting algorithms as needed).


4. App:   Observe how the nurses and doctors interact with the app to identify questions and/or app structures that are pain points or not intuitive, in order to streamline the app accordingly. This could include:

  • updating the   wording   of questions,
  • changing the   format   of certain inputs.


5. User feedback:   Ask the nurses and doctors directly   what the app could be doing better , and if feasible, incorporate that to the app.


6. Content and algorithms:   Look for insight into the content and algorithms in the app, but   we assume you [implementing/content partner] have your own processes   to confirm the rigor of those.


Please note that   is   not an exhaustive list , but highlights of things we will be looking for.


As such, refinement (field) testing is an important part of our design-test-iterate process, which is best done at the ground level by putting an early version of the app in the hands of the end users. We would like to understand the values, objectives and context of the users' reality to create a tool that provides value at the frontline.


Please let me know if you have any questions.