Outcome evaluation of capacity building and mentorship program (CBMP) on data quality in the public health facilities in Amhara National Regional State, Ethiopia

Abstract

Abstract Background: The facility-level health data is a primary source to assess the health sector performance. However, the quality of the health management information system is always in question. The capacity building and mentorship program (CBMP) was launched to improve the health data quality and information utilization on a pilot basis. The outcome of the program is not well known. The finding of this evaluation will be used by policymakers, program implementers, and funding organizations to know the effectiveness of the program and plan for scale-up. Objective: To evaluate the outcome of the capacity building and mentorship program on data quality in the selected public health facilities in Amhara National Regional State, Ethiopia Methods: A matched comparison group evaluation design with a sequential explanatory mixedmethod

was used to evaluate the outcome of CBMP on data quality. The Organization for Economic Cooperation and Development (OECD) evaluation framework with Relevance, Effectiveness, and Impact dimensions was used to measure the outcome of the program. A total of 23 facilities from the intervention group and 17 comparison health facilities from a randomly selected district were used for the evaluation. An in-depth interview was used to assess the relevance of the program and the data was recorded, transcribed, translated, and then thematized for analysis. For effectiveness and impact dimensions a structured questioner was used. The data were entered into EpiData version 3.1 and transferred to Stata for Analysis. The data then presented in frequency, mean, standard deviation, tables, and figures. A t-test, the average treatment effect was estimated. Finally, the result was judged on predetermined criteria. Results: The overall level of CBMP implementation to the judgment parameter was found to be 91.16%. The mean data quality in the intervention health facility was 89.06 [95%CI: 84.23, 93.88]. There was a significant mean difference between the intervention and comparison of health facilities. Besides, the CBMP increase the data quality of pilot facilities by 23.33 % [95%CI: 14.30, 32.35]. The health facility heads and HMIS officers believed that there was a data quality problem in their health facility and think that the CBMP improve the data quality gap. Conclusion: The outcome of the capacity building and mentorship program was highly satisfactory based on the judging criteria. Similarly, the program was effective in achieving its objective and has contributed to the data quality in the pilot health facilities. The program was needed by pilot health facilities and regular mentorship and training of all staff are recommended by the head of the health facilities and health information system officers. Keywords: CBMP, Effectiveness, Outcome, evaluation, Data quality, OECD, Amhara