User Testing

Objective

To enhance the effectiveness of CommCare Users by observing their engagement and understanding, as well as their interaction with patients during visits and training activities. This document also outlines steps for implementing projects with CommCare, emphasizing the importance of preparation, training, observation, and follow-up.

Observation Checklist for CommCare User Engagement

• Assess engagement and understanding levels of CommCare Users and patients.

• Monitor the pace at which CommCare Users proceed through forms.

• Evaluate the clarity of communication, including whether CommCare Users provide additional context beyond verbatim questions.

• Check the audibility of audio features and the visibility of images shown to patients.

• Observe CommCare User’s navigation and data entry accuracy within the app.

• Note any language comprehension issues and confirm the appropriateness of content and language used.

Implementation Steps

1. Background Understanding: Gather all existing documents and understand the operational field by contacting project managers and CommCare User representatives.

2. Focus Group Discussions (FGDs): Conduct at least three FGDs to understand current processes, data flow, and reporting methods. Validate understandings with users and plan field visits for direct observation.

3. Observation: Observe CommCare Users in action, focusing on their interaction with clients without intervening or correcting their methods. Document observations, including the sequence of activities and time taken for services.

4. Specific Issues to Note:

   • Language suitability and comprehension.

   • Efficiency of date-related questions and calculations.

   • Appropriateness and sequencing of questions.

   • Engagement level of patients throughout the interaction.

Training Guidelines

• Train no more than 10 users per session, aiming for 16 hours over 2 days.

• Review and update training materials, ensuring representation from partner organizations.

• Start with basic phone functionalities before introducing the CommCare application.

• Identify quick and slow learners for focused attention and potential peer support.

• Use practical exercises and demonstrations rather than immediately distributing manuals.

Visits and Follow-ups

• Conduct weekly follow-ups for the first month, then bi-weekly if possible, to identify challenges and gather suggestions.

• Use feedback to pair slow and quick learners for peer support.

• Monitor user activity via the CommCare HQ dashboard, encouraging practice and addressing any issues.

• Evaluate the application’s impact on service delivery compared to traditional methods and gather feedback for improvements.

Conclusion

This comprehensive approach aims to maximize the efficiency and impact of CommCare Users, focusing on thorough preparation, engaging training, meticulous observation, and continuous support and feedback.