Malaria rapid diagnostic kits
As a complement to a CDC study in Malawi, QAP developed and implemented a research protocol for testing and modifying job aids (instructional inserts) that accompanied two Malaria Rapid Diagnostic Kits. The study used quality design techniques to improve the job aids and to make recommendations for product design changes. The technique involved observing kits being used, noting problems, redesigning the instructions, having users test them while under observation, and redesigning them again. In an initial test prior to the modification of the instructional inserts, only 3 of 19 (15%) providers used the two kits without making errors. Eight of the providers had received training in the use of the kits, and two of these (25%) were error free. After the instructions were revised, 85% (17 of 20) used the kits without error. The study demonstrated that large improvements can be achieved in the instructions by using the redesign process and that improved inserts can be more effective than training.
Download report:
Using Quality Design to Improve Malaria Rapid Diagnostic Tests in Malawi
Mali
Impact of self-assessment and peer feedback on health worker IMCI performance in Mali
This study examined the effect of self-assessment and peer feedback on the quality of care provided to children with fever in a peri-urban area of Mali. Quality was defined as the percentage of assessment and counseling tasks performed according to nationally accepted standards. A sample of 36 providers was randomly assigned to a program group (12) and a control group (24), and then observed managing 103 cases of child fever (about 3 cases per provider). The program group received the self-assessment and peer feedback interventions, while the control group did not. In the program group, 54% of the tasks were performed according to standard, compared to only 44% in the control, a statistically significant difference. However, users found the self-assessment to be time-consuming and burdensome.
Download report:
The Impact of Self-Assessment with Peer Feedback on Health Provider Performance in Mali
Return to Research by Country page 1, (List by Country)
Back to top
Mexico
Supportive supervision and self-assessment
This study introduced a program of supportive supervision and physician self-assessment to improve provider-client communication with 60 doctors working for Mexico's Social Security hospitals. The intervention was evaluated using audio-taped counseling sessions obtained pre and post in a program and a control group. The study found that facilitative and informative communications per counseling session by the doctors in the program group were approximately twice that in the control group following the intervention; the difference between program and control was statistically significant even after controlling for purpose of visit, sex of doctor, and duration of session. In a pre-post panel analysis of 28 doctors, substantial improvements occurred in both the program and control groups, but the increases were markedly larger in the program group, where increases were about twice as great in provider facilitative and informational communications. Active patient communication, as measured by the number of questions asked by the client, increased dramatically and by the same amount in both the program and control group (from 2.4 to 12.7 questions per session), a finding consistent with the post-intervention result for 60 doctors where no significant difference was observed in active patient communication. The fact that session duration increased pre to post but was not significantly different in program and control indicates that the rate of provider communication went up in the program group. The self-assessment experience by the supervisors and the doctors was highly regarded by them, as was use of tape recorders by the doctors to listen to their own counseling sessions.
Download report:
Participatory Supervision with Provider Self-Assessment Improves Doctor-Patient Communication in Rural Mexico
Morocco
Factors influencing success of quality improvement teams
A sample of 23 quality improvement (QI) teams from 27 health facilities (9 hospitals, 18 health centers) in four regions of Morocco were studied to determine the effect of coaching and functionality on their success. Functionality was defined as the degree to which the team followed a simplified 4-step QI problem-solving protocol (identify problems, analyze and select the key problem, develop and implement solution, and define and measure monitoring indicators). Data were obtained about the number of problems defined, problem-cycles completed, and success in implementation and problem solving through structured questionnaires and interviews with team members, team coaches, and regional QA directors. Success for a particular problem-solving cycle was defined by whether the respondents said the team accomplished or not the chosen objective and by whether the monitoring indicator showed improvement, no change, or decline. With respect to functionality, teams did well on problem identification (80%) and problem analysis (90%), but poorly on solution development and implementation (55%). Of the 14 teams that completed at least one problem-solving cycle, 67% engaged in good monitoring techniques, and 80% reported achieving improvement in their indicator of interest. This suggests that functionality is related to success as measured.
Download report:
Quality Improvement Teams in Morocco: An Evaluation of Functionality and Success
Return to Research by Country page 1, (List by Country)
Back to top
Niger
Client satisfaction
This study examined the utility, validity, feasibility, and cost of several types of client satisfaction data collection methods in three districts of Niger. Patient satisfaction data were collected from exit interviews with 301 clients and from 24 client focus groups. Three types of data collectors were used: health supervisors from the same district, outside enumerators, and health supervisors from a neighboring district. The advantages and disadvantages of the two data collection methods and three data collector types were measured using key informant interviews with district managers, questionnaires to the data collection teams, cost records, and analysis of the exit interviews and focus group data. The study found that exit interviews were more valid and useful than focus group data and were also less costly in terms of data collector time. Supervisors from the same district were less costly, more useful, but slightly less valid than the alternatives. Outside enumerators may be more feasible by not drawing on overworked supervisors.
Download report:
Helping District Teams Measure and Act on Client Satisfaction Data in Niger
Compliance, workload, and cost of IMCI in Niger
QAP studied the relationship between compliance with guidelines for the Integrated Management of Childhood Illness (IMCI), cost (drugs and provider time), and workload for 211 child cases of fever, cough, diarrhea, and earache in 26 health clinics in Niger. The study found rates of compliance with IMCI guidelines of 33 percent for assessment tasks, 81 percent for treatment, and 42 percent for counseling. No relationship was found between compliance and cost, even for particular diagnoses, nor between average clinic compliance and average clinic workload. While previously published studies have reported that increased compliance with IMCI guidelines has resulted in lower drug costs, this study suggests that such a relationship is not always present. The study also found that increased compliance did not lead to increased costs, either for drugs or for consultation time.
Download report:
Compliance, Workload, and the Cost of Using the Integrated Management of Childhood Illness Algorithm in Niger
Impact of QA on IMCI compliance in Niger
This study examined the effect of performance feedback and feedback plus IMCI training on provider compliance with IMCI guidelines in three districts of Niger. Compliance with assessment, treatment, and counseling standards was measured by direct observation of IMCI cases at four different times over 18 months. The results varied from standard to standard and over time, in ways that are difficult to interpret. Nevertheless, there appears to be at least a short-term positive impact of IMCI training and, to a lesser extent, of performance feedback on assessment compliance. Training plus feedback had the most impact, increasing a 17-task composite index by 27 percentage points (from 21% to 48%) six months after the interventions. Indicator measurement and performance feedback cost $108 per health worker over the course of the study, and IMCI training cost $430 per worker.
Download report:
Impact of QA Methods on Compliance with the Integrated Management of Childhood Illness Algorithm in Niger
Job aids to improve adherence to cotrimoxazole treatment in Niger
This study tested the effect of a set of job aids and provider training on provider, caretaker, and patient adherence to cotrimoxazole for the treatment of childhood pneumonia in Niger. Data on 677 cases of childhood pneumonia were obtained from home and clinic observations and interviews in 4 experimental and 4 control clinics following the introduction of the job aids in the experimental clinics. The study found that caretaker adherence to the recommended regimen was much higher in the experimental than control clinics for cases treated by technical nurses (lower educational level), while little effect was observed on professional nurses (higher educational level). In fact, the performance of technical nurses in the experimental clinics rose to near the performance of the professional nurses, suggesting that the job aids may be a low-cost alternative to more expensive formal education. Another interesting result was that providers should prescribe and give caretakers the full course of pills at the initial visit. Partway through the study it was discovered that the clinics were giving a partial course of cotrimoxazole at the initial visit (1 to 3 days of pills rather than 5 days) for a variety of stated reasons (e.g., force caretaker to return for follow-up visit, misunderstood recommended practice, low inventory of pills). In fact, the study showed that children were far more likely to consume the full course if their caretaker received it at the initial visit than if not.
Download report:
Improving Adherence to Cotrimoxazole for the Treatment of Childhood Pneumonia in Niger
The Niger QAP/BASICS Joint Project Final Evaluation
QAP launched a pilot program in Niger in 1993 to strengthen newly decentralized regional and district management and improve the quality of primary healthcare in the regional Department of Tahoua. The project sought to develop the capacity of regional and district management teams to apply quality management (QM) techniques to improve the quality and client-focus of basic services, strengthen teamwork, and improve the measurement and monitoring of results. The Basic Support for Institutionalizing Child Survival Project (BASICS) became active in Niger in 1995, helping to improve child health services, especially the care of sick children. In 1997, the two projects merged in Niger to form a joint project, QAP/BASICS. The final evaluation of the joint project was conducted in 1998 and had two objectives: 1) to assess the degree of quality assurance institutionalization in the Tahoua Department as a result of the work of the Tahoua Quality Assurance Project since 1993, and 2) to summarize lessons from introducing the Integrated Management of Childhood Illness (IMCI) guidelines in a quality management (QM) environment. The evaluation team collected data from the two departments of Niger in which the joint project was carried out: Tahoua and Dosso. The team visited the two departmental headquarters, six district headquarters, and 13 integrated health centers. These represent 80% of the facilities involved in the project. Using structured questionnaires, the team interviewed approximately 60 staff members, covering all levels of the healthcare system in both departments, as well as international partners (BASICS, WHO, UNICEF, and German Cooperation representatives). To measure IMCI performance, the team reviewed a sample of completed IMCI forms in three facilities each in Boboye and Konni Districts. The evaluation team also organized two focus groups of women who had used prenatal care and child health services in Tahoua. The team prepared three case studies to document quality improvement processes and results in three facilities.
Download report:
The Niger QAP/BASICS Joint Project Final Evaluation
The sustainability of quality assurance activities in Tahoua, Niger
This study assessed the continuation of quality assurance activities in the Tahoua region of Niger two years after the close of the QAP/BASICS project. The project, which operated from 1993 to 1998, provided technical and operational support to basic health services in order to improve their quality through training, supervision, meetings, operational research and provision of small equipment. This study compared current QA activities in 8 districts of Tahoua and 4 districts of a control region (Maradi) during the two years (1999 and 2000) after the QAP/BASICS project closed. Interviews of Niger Ministry of Public Health personnel and review of documents were conducted at all levels in the health system. Healthcare workers were observed to determine compliance with IMCI standards and caretakers were interviewed after the observation. In the last two years in Tahoua, more standards were revised than in Maradi (4 vs. 2), and more district coordination meetings occurred in 2000 (44 vs. 32) but not in 1999 (56 vs. 58). The percent of planned supervisory visits actually completed was higher in Tahoua than in Maradi in 1999 (51 vs. 24) but not in 2000 (52 vs. 55). The number of persons trained in QA continued at approximately the same level after the close of the project (22 in 1997, 79 in 1998, 26 in 1999, and 76 in 2000), but the number of problem-solving cycles completed by QA teams decreased after project close (31 in 1998, 21 in 1999, and 17 in 2000). Several factors helped maintain the project's accomplishments: new funding, availability of trained staff, demonstrated results, and staff commitment. However, the regularity of supervision and the QA team productivity suffered after the project closed.
Return to Research by Country page 1, (List by Country)
Back to top
Peru
Activity-based costing at MaxSalud clinics in Peru
Activity-based costing (ABC) allocates indirect costs to outputs according to the actual work done to produce the outputs rather than in proportion to the relative volume or direct costs of outputs as in traditional cost accounting. This study applied ABC to calculate the unit cost of services provided by MaxSalud clinics in Peru. The application included: (1) the description of all departments, services, and activities by department, (2) staff estimates of time spent on each activity and unproductive time, (3) estimated cost of all activities by each department using wage and other data, (4) trace activities and costs within and across departments to services provided, and (5) estimated service volumes from records and calculate unit costs (cost/volume). The study identified 107 distinct activities at MaxSalud, including training and meetings. ABC derived unit costs that were generally higher than prior estimates and much higher than fees charged. The study concluded that ABC is potentially very valuable to MaxSalud to set policy, manage expenditures, and even raise funds, but it requires reliable data systems for costs and service statistics, management attention, staff support, and technical assistance.
Download report:
Application of Activity-Based Costing (ABC) in a Peruvian Based NGO System
Measuring client satisfaction at MaxSalud clinics
This study examined the advantages and disadvantages of six different methods for obtaining client satisfaction information at the MaxSalud clinics in Peru. The methods included: client exit interviews (323 clients), client home follow-up visits (32 visits), focus groups (8), home interviews with discontinued clients (40 visits), suggestion boxes, and community meetings. Exit interviews were relatively inexpensive, quantitative, periodic, and useful for ongoing monitoring. Follow-up visits provided valid, in-depth information useful for quality assurance work. Focus groups were useful but not very feasible. Discontinued client interviews were less useful and relatively expensive. Suggestion boxes are easy to establish but don't produce much useful information. Community meetings provide contextual information, not useful client satisfaction data. Clinic managers and quality committees appreciated and used the client satisfaction data, especially when it was succinctly summarized.
Download report:
Implementing a Client Feedback System to Improve the Quality of NGO Healthcare Services in Peru
