This page contains questions to ask or things to look for when someone wants to observe how a CommCare phone user such as a Community Health Worker (CHW) is using CommCare during training activities or during visits. This is supposed to be a running list, so please add more questions.

  • In general, what is level of engagement of both CHW and patient?
  • In general, what is level of understanding of both CHW and patient
  • How quickly is the CHW proceeding through the forms?
  • What appears to be the level of understanding for the patient? Do they look like they understand the questions?
  • Does the patient have short answers?
  • Are the questions simply said verbatim from the phone? Does the CHW provide any additional context or information to make sure the patient is correctly interpreting/understanding the questions?
  • Is the audio easily audible?
  • Is the CHW showing the images on the phone to the patient?
  • Look from behind the CHW to see how they are proceeding through the forms.
  • Does the CHW seem to know how to navigate the application pretty well?
  • Is the CHW entering responses into the phone correctly (yes when client says yes; no when client says no)?

Definition File for In-Field CHW Observation (developed by CRS, includes CHW registration, CHW interpersonal communication checklist, home visit observation checklist, and technical issue tracker)

 

 

The implementation of our projects will contain the following steps;- 

  1. BACKGROUND OF THE PROGRAM AND WHAT IS GOING ON

It is important for the implementer to have a better and full understanding of the background of the project, and it is run currently before we introduce CommCare. To achieve this the implementers is suppose to request the following;-

  • Request all existing documents from our partner organization
  • Get the contact person from the operational field of our partner organization (the manager of the project)
  • Get to know the site where the project is going to be implemented
  • Get the contacts of the users or representative of users who are going to use the tool e.g. CHWs

 

  1. FOCUS GROUP DISCUSSION
  • The implementers is suppose to conduct at least 3 focus group discussion with the users
  • During that time, the implementers is only listening and asking questions to understand how the existing go and data flow and how they report
  • It is good once you have put together a document you should go back to check with users see if you have understood and make corrections according to users
  • Then schedule some field visit so that you can actually go see what they are doing

 

  1. OBSERVATION
  • After you have conducted a focus group discussion with users, it is very important for the implementers to go follow-up with the users to actually see how they work.
  • During observation field visit, remember you are only there to observe what take place. You can ask some things but don't try to correct things that you have heard from the organization or during focus group discussion.
  • Write down a report on what you have seen exactly what happens; if possible include the dialog or interactions between the user and clients.
  • Don't forget how it begins, the questions and answers but also what is been noted down by the user and clients.
  • Determine how much time it takes from when the service begin to the time it ends
  • At this point be carefully on how you talk to the user e.g. CHW because if you try to correct the user or ask question like you are suppose to do this and that, he/she may feel being supervised which will affect the entire point of you being there.
  • Tell the user that you are there because you want to learn how they work and try to appreciate what the person is going.
  • After the exercise is over you may now ask question like oh! I didn't see you do this or that would you tell me why? At that time there is no right and wrong answer but you are interest is to find out why users may or may not do this or that as the organization expects them to do
  • Assure the users that you are not accusing them but you are trying to understand the difference between their services. At that point we don't know whether that's a good way or bad.

    1. Specific issues 

Specific things to pay attention to/observe during a field visit include the following. You may want to keep two lists: (1) improvements to make to the application and (2) points to include/emphasize during training and in any manuals.  

  • Language: Confirm that user is comfortable with whatever language has been chosen to be displayed on the phone, whatever language they are asked to enter information on the phone, and whatever language the audio message is being played in. Similarly, confirm that the audio message is being understood by the household.
    • If there are comprehension issues, are they related to the wrong language being used? Or to the literacy level of the user/beneficiary having been over-estimated? 
  • Dates: What is the best/most usable way to pose a date question - ask for the date? how long ago it was? how old a person is or was then? Is date information typically obtained from the person or via reference to some kind of health card? 
  • Calculations: Is the user spending time doing any unnecessary calculations that the phone could handle? 
  • Propriety: Are all the questions appropriate? Should sensitive questions be placed later on in the interview? (e.g. after building more rapport or after curious observers have gotten bored and wandered away) 
  • Workflow: Does the sequencing of questions make sense?
  • Length: Time the interview. If you think the timing of the interview observed is reasonably representative, does the beneficiary seem too tired/listless at the end? 
  • Audience: Who is present when the interview is happening? Is there a way for the app content to take this into account? Should it?
  • Answerability: Is the beneficiary capable of answering the questions she is posed? (e.g. if she is at a clinic, she may not be able to go and find out what her household's phone number is. She may not be able to answer questions about her medical history or family income.) 

  1. Training guidelines

This guidelines is for both community and facility based, basically trying to give an idea of how much resources is needed during implementation.

  • Maximum number of users to be trained by one person or implementers is 10
  • The period of training should be at least 2 days with 8 hours each so a total of 16 hours
  • Before the implementer plan the training, it is important to review the training material one week before. Cross check with the application to make sure it is updated
  • Arrange with our partner organization to make sure they send a representative
  • Communicate to the users and make arrangement for the venue where the event will take place.
  • Put the updated training material on dropbox together with the new jar file
  • On the training day, greet users and make some stories that will make them interested on you and on the event.
  • Ask how many of them have ever used a phone before, don't be shocked if many people have never touched a phone before
  • Encourage those who have never used a phone before that don't worry you will get there just listen and put more effort during the training
  • If possible pair them up by having one person who is familiar with the phone and the one who is not familiar
  • Introduce the model of the phone you are going to use and be careful to remind those who have used before that even if they know they should go slowly step by step
  • Don't begin training them on the application
  • Start with the basic things such how to turn on and off, how to scroll up and down and how you navigate through the main menu.
  • Show them how to write and send sms
  • How to set date and time
  • How to increase/decrease tone and call volume
  • How to navigate through the phones using different buttons
  • Give them few translations of key words during navigation such as delete, back, exit, select, options etc
  • Once they are excited and there will be notice in the room everyone asking and talking. Let them do that for about 10 – 15 minutes, at that time don't just sit down but walk around the room and try to identify which users are smart and understand quickly. Also which users are learning slowly.
  • The success of your training is at this time so don't miss your opportunity make a note of all slow learners. Because during training they need your special attention. It is ok if you could pair them up again to make sure they get help from super users.
  • After this exercise now you can introduce the application by showing
  • How to open CommCare application
  • How to use Demo user and why and what time should we use DEMO mode
  • How to use Login screen and password, when should they login using password
  • Gave demonstration on the effect of interchanging Demo user to the login using password. Where we told them DEMO is for practice while login is for household and client records.
  • During this time don't hand out the training manual, let them use phones step by step.
  • You can hand out the training manual at the end of the first day, this will help them practice more in the evening and let them come up with many questions on the second day
  • Keep the training materials updated on the dropbox

 

  1. Field visit and follow-ups
  • The follow-up should take place in the next 4 weeks, once per week at least four hours per week.
  • The implementers should be able to get feedback from each user identify challenges and suggestions
  • It may be a good time to identify the users who are learning slowly and those who are learning very fast. If you identify someone who is smart leaning we can let them train others or pair them with someone who doesn't learn as quick. (Power testers)
  • During that time encourage the users to practice and if possible send them an sms during the week before you see them. Or call each of them to encourage them
  • After four weeks of follow-up you can space and start meeting them once after two weeks. It is not necessary but if you can do this at least twice may help get people to full speed.
  • The implementers should be able to login on the CCHQ, look on the dashboard and call/SMS those who have not submitted forms. It is ok for them to use a little bit of money to text us if they have a problem. If necessary for the implementers to visit the field can schedule that.
  • During that time schedule some meetings with clients to see how they feel about the application and compared to the old system.
  • Also have their feedback on how it can be improved
  • It may be helpful to come up with a questionnaire during that time especially if some of deliverables need the info from the clients.
  • Whether with or without questionnaire you should be able to write down the challenges and success stories.
  • Write down frequent asked questions from the meeting

  File Modified
Microsoft Excel Sheet CommCare_definition_SF_TrackingTools_18 July 2012.xls Nov 25, 2012 by Mohini Bhavsar
Microsoft Excel Sheet CommCare_definition_SF_TrackingTools_19 July 2012.xls Nov 25, 2012 by Mohini Bhavsar
Microsoft Word 97 Document Form Improvement Process Survey_v2.doc Oct 31, 2013 by Jeremy Wacksman
Microsoft Word Document BHOMA CHW Monthly Survey.docx Oct 31, 2013 by Jeremy Wacksman
Microsoft Word Document Field Testing Checklist-v3-mb (generic).docx Apr 07, 2014 by Mohini Bhavsar
Microsoft Word Document Goals of Refinement (Field) Testing.docx Dec 12, 2014 by Kriti Mehrotra

  • No labels