Tools to assess quality and experience of care provided by community health workers using mobile technologies

Introduction

Community Health Workers form an integral part of public health systems in developing countries, and are often the first point of contact for maternal and child health services in rural areas. Most research studying the effectiveness of community health programs assess health outcomes to analyze the impacts of community health programs. Access, quality, and experience are the three factors that lead to improvements in health outcomes. Some studies do focus on improvements in quality or experience where community health workers are concerned, and improving the quality and experience of care provided by CHWs is an effective way to generate gains in health outcomes.

 

Currently, there is a lack of tools to assess the quality and experience of care provided by community health workers. As with other community health programs, community health programs utilizing m-health technologies, do so with the aim of improving health outcomes, via improvements in access, quality, and experience of care. However, tools to measure quality and experience of health care provided by CHWs utilizing m-health technologies have not been developed and rigorously tested. A tool to directly assess quality and experience of care, will facilitate better understanding of the use and impacts of m-health on the health services being received by the communities.

 

We have developed a tool to assess the quality and experience of health care provided by CHWs using m-health technologies. Programs utilizing m-health with the aim of improving quality and experience of health care provided by CHWs can use the tool to evaluate and assess the impacts of their program. Furthermore, improvements can be made to the tool after it has been disseminated and scrutinized widely.

 

The objective of our study is to develop, present, and discuss tools to assess quality and experience of care provided by community health workers using m-health platforms to provide maternal and new born care to their communities. Our tools can also be used to assess the quality and experience of health care outside of maternal and new born care.

 

METHODOLOGY

 

The preliminary tool was developed based on common features of an m-health application, specifically, CommCare. The tool was field tested, and modified with 2 rounds of modifications after field testing to capture all the necessary elements. Field testing involved shadowing the community health worker on home visits and utilizing the tool to observe specific elements of the home visit meant to capture the quality and experience.

 

 

THE TOOL

The tool has two parts, where we first assess the CHW’s proficiency and use of Commare, and second focus on observing the home visit to capture quality and experience of Care.

The exact components that go into measuring proficiency, quality and experience are described below.

CommCare Proficiency and Use

CommCare Proficiency is measured directly during the home visit assessment as a composite score based on whether the ASHA can perform certain tasks in CommCare, and whether or not she used certain features of CommCare. We identify a) navigation in the phone and inside the application; b) ability to select the client from a list of registered patients; c) use of all the forms listed for the visit; d) entering accurate answers; e) entering text, dates, and numbers; and f) ability to play audio and video, as features that measure CommCare Proficiency and Use.

ASHAs can earn a maximum of 16 points for CommCare Proficiency and Use.

Quality of Home Visit

We measure the quality of the home visit through a quality score generated by scoring the visit based on a) whether all the forms listed were used during the visit; b) whether the visit included counseling; c) the number of counseling topics, and for each topic if 1) complete and accurate information was provided, 2) if the client asked questions, and 3) if the ASHA verified that the messages were received. ASHAs can receive a maximum of 14 points for Quality Score.

We also include a second measure for visit quality based on the researcher’s perception of the home visit. This is a subjective measure of the visit quality, classified again as Low, Middle, or High, based on the researcher’s perception.

 

Experience of Home Visit

 

Experience of home visit is also measured directly during the home visit assessment. It is a composite score, generated by adding scores regarding the frequency of a) audio usage, b) video usage, c) showing images, d) whether all of the people present were addressed, e) whether the ASHA spoke clearly and loudly with confidence, and f) the duration of the visit, where under ten minutes scored a 1, 10-20 minutes scored 2, and over 20 minutes scored 3. ASHAs can receive a maximum of 12 points for the experience score. The rationale for measuring visit experience in this way starts with the assumption that the use of multimedia increases the quality of the visit (Treatman and Lesh, 2012).

Annex 1: CHW Home Visit Observation Form: CommCare

 

 

 

 

 

 

CHW Information

 

1

CHW Name

 

 

2

Village

 

 

3

Block

 

 

4

Phone Number

 

 

 

Client Information

 

5

Client Name

 

 

6

Type of Client

Pregnant Mother

Lactating Mother

 

7

Months

(number of months the client is pregnant or lactating)

 

 

 

8

Village

 

 

9

Block

 

 

 

Visit Information                    (Scores are in parenthesis below the answer options)

 

10

Date of Visit

 

 

11

Time of Visit Start

 

Use 24hr clock

12

Time Visit End

 

Use 24hr clock

13

Duration of Visit:

( 1-Under 10 min, 2- between 10 and 20 mins, 3-over 20 mins)

 

Minutes

14

Number of people present

  Adults

Children

 

14.1

Is the mother in law present?

Yes

No

 

14.2

What attracted her to the session?
( if 14.1 is Yes)

 

 

 

 

 

Observation Checklist: Use of CommCare                 ( This is used to create a CommCare Proficiency Score)

Starting and Navigation                               (Scores are in parenthesis below the answer options)

15

Did the ASHA start CommCare without being prompted

Yes
(1)

No
(0)

16

Did the ASHA have trouble opening Commcare?

Yes

(0)

No

(1)

17

Did the ASHA have trouble logging in?

Yes

(0)

No

(1)

18

Did the ASHA have trouble navigating inside the forms?

Yes, had trouble all the time

(0)

Yes, had trouble sometimes
(0.5)

No, did not have trouble

(1)

19

Did the ASHA have trouble navigating between forms?

Yes, had trouble all the time

(0)

Yes, had trouble sometimes
(0.5)

No, did not have trouble
(1)

Content of the Forms                                   (Scores are in parenthesis below the answer options)

20

Did the ASHA have trouble choosing the type of form to use?

Yes, had trouble all the time
(0)

Yes, had trouble sometimes

(0.5)

No, did not have trouble

(1)

21

Did the ASHA have trouble finding the client in CommCare?

Yes, had trouble all the time

(0)

Yes, had trouble sometimes

(0.5)

No, did not have trouble
(1)

22

Did the ASHA view all the case details?

Yes

(1)

No

(0)

23

Did the ASHA use all of the listed forms for that visit?

Yes

(1)

No

(0)

24

Did the ASHA answer all relevant questions in the forms only after asking the client?

Yes, all the time

(1)

Yes, sometimes
(0.5)

Never
(0)

25

Did the ASHA skip any questions in the form?

Yes, skipped a lot of questions

(0)

Yes, skipped a few questions

(0.5)

Did not skip any questions

(1)

26

Did the ASHA provide counseling on all the topics listed in the form?

Yes

(1)

No
(0)

27

Did the ASHA record accurate information as relayed by the client for each question?

Yes

(1)

No

(0)

28

Did the ASHA have trouble entering text?

Yes

(0)

No

(1)

Not Applicable

(.)

29

Did the ASHA have trouble entering numbers?

Yes

(0)

No

(1)

Not Applicable

(.)

30

Did the ASHA have trouble entering dates?

Yes

(0)

No

(1)

Not Applicable

(.)

Media                                                               (Scores are in parenthesis below the answer options)

31

Did the ASHA show the images in CommCare to the client?

Yes
(1)

No

(0)

31.1

How often did the ASHA show images to the client?

Showed Each Image

(1)

Showed some of the images

(0.5)

Did not show any image

(0)

32

Which videos were prompted in the form for this visit?

Birth Preparedness

 

Family Planning

Cord Care

Complementary Feeding

32.1

Which videos did the ASHA show the client?

Birth Preparedness

Family Planning

Cord Care

Complementary Feeding

33

Did the ASHA play the audio available in CommCare?

Yes

(1)

No

(0)

33.1

How often did the ASHA use the audio available in CommCare?

Played audio for each question

(1)

Played audio for some of the questions

(0.5)

Did not play any of the audio

(1)

 

 

Perception of Visit                                 (Scores are in parenthesis below the answer options)

34.1

List all the forms that the ASHA used for this visit?

Form1:

.2

 

Form 2:

34.3

 

Form 3:

35

Did the visit include any counseling?

Yes
(1)

No
(0)

36.1

What were the main counseling topics of this visit? List up to three

Topic 1:

36.2

 

Topic 2:

36.3

 

Topic 3:

Topic 1
(1)

37a

Did the ASHA provide complete information to the client on each topic listed?

 

Yes
(1)

No
(0)

38a

Did the ASHA provide any inaccurate information?

Yes
(0)

No
(1)

39a

Did the client ask the ASHA any questions?

No did not ask any questions (0)

Yes some questions
(0.5)

Yes asked a lot of questions
(1)

40a

Did the ASHA verify that the client is understanding the messages being delivered?

No did not verify

(0)

Yes, after some of the messages
(0.5)

Yes after all of the messages (1)

Topic Two:
(1)

37b

Did the ASHA provide complete information to the client on each topic listed?

 

Yes
(1)

No
(0)

38b

Did the ASHA provide any inaccurate information?

Yes
(0)

No
(1)

39b

Did the client ask the ASHA any questions?

No did not ask any questions (0)

Yes some questions
(0.5)

Yes, a lot of questions
(1)

40b

Did the ASHA verify that the client is understanding the messages being delivered?

No did not verify

(0)

Yes, after some of the messages
(0.5)

Yes, after each message

(1)

Topic 3:

(1)

37c

Did the ASHA provide complete information to the client on each topic listed?

 

Yes
(1)

No
(0)

38c

Did the ASHA provide any inaccurate information?

Yes
(0)

No
(1)

39c

Did the client ask the ASHA any questions?

No did not ask any questions (0)

Yes some questions
(0.5)

Yes, a lot of questions
(1)

40c

Did the ASHA verify that the client is understanding the messages being delivered?

No did not verify

(0)

Yes, after some of the messages
(0.5)

Yes, after each message

(1)

41

Did the ASHA address all of the people present during the visit?

Yes
(1)

No
(0)

42

Did the ASHA speak clearly and loudly?

Yes
(1)

No
(0)

43

Did the ASHA speak with confidence?

Yes
(1)

No

(0)


 

Additional Observations

44

 

How would you rate the quality of this visit?

High

Medium

Low



45

 

Any other comments?