|Year : 2019 | Volume
| Issue : 1 | Page : 205
Evaluation of timeliness, simplicity, acceptability, and flexibility in child mortality surveillance system for children aged 1–59 months in Iran
Mansour Bahardoust1, Abdolhalim Rajabi2, Seyyed-Hamed Barakati3, Morteza Naserbakht4, Shila Ghadami5, Elham Talachian6, Seyed Abbas Motevalian7
1 Department of Epidemiology, School of Public Health, Iran University of Medical Sciences, Tehran, Iran
2 Department of Biostatistics and Epidemiology, School of Public Health, Golestan University of Medical Sciences, Gorgan, Iran
3 Child Health Office, Ministry of Health and Medical Education, Tehran, Iran
4 Mental Health Research Center, Tehran Institute of Psychiatry- School of Behavioral Sciences and Mental Health, Iran University of Medical Sciences, Tehran, Iran
5 Family and Population Health Office, Ministry of Health and Medical Education, Tehran, Iran
6 Pediatric Gastroenterology Division, Ali-Asghar Children's Hospital, Iran University of Medical Sciences, Tehran, Iran
7 Research Center for Addiction and Risky Behaviors (ReCARB), Psychosocial Health Research Institute (PHRI), Iran University of Medical Sciences (IUMS), Tehran, Iran
|Date of Submission||06-Oct-2018|
|Date of Acceptance||05-Oct-2019|
|Date of Web Publication||28-Nov-2019|
Seyed Abbas Motevalian
Research Center for Addiction and Risky Behaviors (ReCARB), Psychosocial Health Research Institute (PHRI), Iran University of Medical Sciences (IUMS), Tehran
Source of Support: None, Conflict of Interest: None
Background: Child mortality surveillance system (CMSS) for children aged 1–59 months is a critical issue in the prevention of mortality. This surveillance system like other health programs needs to be evaluated. Therefore, this study aims to evaluate CMSS in Iran. Methods: This evaluation was performed from March 2015 to March 2016 based on selected criteria for assessing the public health surveillance system proposed by the Centers for Disease Control and Prevention. Selected criteria examined in this study included timeliness, simplicity, acceptability, and flexibility. These criteria were evaluated in two ways. First, it included the use of a researcher-made questionnaire. The questionnaires were completed by 100 experts on CMSS. Second, to perform a more exact evaluation of these criteria, 24 of these experts were selected for the focus group. Results: In this study, the response rate was 91% (42% hospital-based and 49% primary care-based). In the timeliness section, 49% of the experts believed that approvals of the child mortality committees have not been sent within the designated time frame; hardware, software, and questionnaires were reported as effective factors in this respect. The structural and administrative problems were effective in simplicity domain and the experts of mortality registration and mood of relatives were effective in acceptability domain. The flexibility of the system was high and appropriate. Conclusions: The findings of the present study reveal that CMSS has some limitations and problems in the timeliness, simplicity, and acceptability criteria, which can be resolved. But this program has an appropriate situation in terms of flexibility.
Keywords: Child mortality, program evaluation, public health surveillance, Iran
|How to cite this article:|
Bahardoust M, Rajabi A, Barakati SH, Naserbakht M, Ghadami S, Talachian E, Motevalian SA. Evaluation of timeliness, simplicity, acceptability, and flexibility in child mortality surveillance system for children aged 1–59 months in Iran. Int J Prev Med 2019;10:205
|How to cite this URL:|
Bahardoust M, Rajabi A, Barakati SH, Naserbakht M, Ghadami S, Talachian E, Motevalian SA. Evaluation of timeliness, simplicity, acceptability, and flexibility in child mortality surveillance system for children aged 1–59 months in Iran. Int J Prev Med [serial online] 2019 [cited 2020 Mar 30];10:205. Available from: http://www.ijpvmjournal.net/text.asp?2019/10/1/205/271852
| Introduction|| |
The child mortality surveillance system (CMSS) for children aged 1–59 months in Iran started nationally in 2007 onwards across Iran's medical universities with the goal of reducing the mortality rate of children. To achieve this goal, the system uses appropriate interventions to review the history of each case of child death from the onset of the first symptom of danger until death and to identify affecting and intervening factors. This system was implemented as a hospital surveillance system and as the primary health surveillance system (titled “out of hospital”). In general, the CMSS consists of four main activities: 1) collection of children's mortality data; 2) examining the causes of death; 3) design and implementation of interventions; 4) monitoring and evaluation. Children's mortality is an indicator of the population health, which is associated with various factors such as socioeconomic status, maternal health, quality and access to health services, and public health functions., According to the eight Millennium Development Goals, committed by all countries in 2000, the child mortality rate should be reduced by 75% in 2015 as compared with that in 1990. Surveillance helps in detecting diseases, supporting health organizations, and interventions, estimating the impact of a disease or injury and portraying the natural history of diseases.,
Evaluation of a public health surveillance system gives a clear description of the purpose, design, management, and operational characteristics of the system and provides evidence-based information, which could be used for strengthening the reporting mechanism and enhancing implementation of public health action. Therefore, CMSS examines the history of each child's death from the onset of the first symptom of danger to death in terms of quality, quantity, and ease of access. It also identifies effective factors in child mortality and designs appropriate interventions for it. Therefore, regular evaluation of the information provided by the surveillance system is essential.,,
The assessment of the CMSS for children aged 1–59 months in Iran was carried out using selected indicators of the Centers for Disease Control and Prevention (CDC), which included simplicity, flexibility, acceptability, and timeliness so that it shows the current status of the CMSS to national authorities and policymakers and, if necessary, immediate interventions can be designed to improve the characteristics of the CMSS.
| Methods|| |
This study was approved by the Vice Chancellor at the Research and Ethics Committee for Research of Iran University of Medical Sciences with number code 26212/2015.
In this cross-sectional survey, definitions, goals, and applications of CMSS in Iran were first summarized, and then the evaluation of the CMSS was performed in 2014 based on selected criteria for assessing the public health surveillance system proposed by the CDC. Selected criteria examined in this study included timeliness, simplicity, acceptability, and flexibility. These criteria were evaluated in two ways. The first one included the use of a researcher-made questionnaire. The questionnaires were completed by 100 experts on CMSS for children aged 1–59 months in Iran. The experts were randomly selected from those participating in an annual conference on “Prevention of child injury and death.” The components of the questionnaire included questions about the timeliness at each stage, simplicity, flexibility, and acceptability of the surveillance program. The questionnaire was designed in in-hospital and out-of-hospital parts. Responses on a five-point Likert scale (I totally agree, agree, have no idea, disagree, completely disagree or very simple, simple, so-so, difficult, very difficult). The reliability and content validity of the questionnaire were examined for these domains. The face validity was evaluated by pediatricians, experts working in CMSS of Ministry of Health, and Medical Education of Iran and epidemiologists to find the level of difficulty, the degree of inappropriateness, and the ambiguity in the expressions or in the meaning of the words. Their views were applied as minor changes in the questionnaire. The reliability of the questionnaire was also evaluated using Cronbach's alpha method for each domain. Cronbach's alpha coefficients of 87%, 94%, 95%, 91%, and 81% were obtained for in-hospital and out-of-hospital timeliness, simplicity, flexibility, and acceptability, respectively.
The second part: To perform a more exact evaluation of these criteria, the focus group was formed using experts in the CMSS. This focus group participated in two groups of 12 people. The participants in each group were later asked questions on the qualitative features of the surveillance system, including timeliness, simplicity, flexibility, and acceptability.
The data of the questionnaires were entered into SPSS software (version 22), and the frequency distribution was calculated for each item of timeliness, simplicity, acceptability, and flexibility. The content analysis method was used to extract the theme, category, subcategories, and code of the information collected from the focus group. The views of 24 experts of CMSS were reviewed and summarized and a single text (a unit of analysis) was later formed. Then, the summarized text was separated into units of meaning. And, they were eventually compared according to their differences and similarities and classified into subcategories and categories. Initial categories were discussed by the researchers. Differences eventually led to an agreement on the categorization of subcategories, and the main meaning and theme were finally constituted within a theme by categories.
Questionnaire A: It was designed to investigate the general specifications of family and health records. Questionnaire B: It was designed to review the medical outpatient services provided to a deceased patient. Questionnaire C: It was designed to investigate records of medical procedures. Questionnaire D: It was a checklist for investigating the causes of death.
| Results|| |
In the timeliness domain of the out-of-hospital CMSS, 49% of the experts believed that the instructions of the CMSS review committees would not be sent to the executive level at a given time and this process would take longer time. Overall, out-of-hospital experts believed that timeliness set by the CMSS for reporting is far beyond the time limit. The main problem of the in-hospital CMSS, according to experts of this system, is related to holding a monthly committee to monitor child mortality; 56% of experts believed that such a committee is not held monthly [Table 1]. Regarding the timeliness domain, results showed that most of the experts in the focus group pointed out the structural and implementation problems of the surveillance program for the expected time period. Therefore, the expected time (24 hours) for the initial registration of a child's death is very limited. According to the experts' opinion, there are some problems that cause limitation including hardware problems of the surveillance program (e.g., old computers and lack of appropriate computers), software problems (e.g., lack of access to the Internet and lack of timely access to CSO software), budget and financial problems, high volume of questionnaires and lack of timely cooperation of the deceased's family to receive the deceased's information; also, a waiting time of 24 hours has significantly limited the reporting and recording of the death cases.
|Table 1: Distribution of experts surveillance system's response to the timeliness in out-of-hospital and in-hospital child mortality surveillance system (CMSS)|
Click here to view
Different items of simplicity domain of the CMSS were evaluated and the results showed that 54.58% of the experts have referred to the incomprehensibility of “E” questionnaire directions in the surveillance system as the main problem. Also, 48.35% of surveillance system experts referred to the difficulty in distinguishing between the main cause and the secondary cause of death as one of the major problems in this domain of the CMSS, on the basis of a questionnaire survey [Table 2]. In addition, the experts of the focus group of CMSS indicated that the surveillance system experts pointed to structural problems of the program and educational problems as the main problems of the system. Also, the problems with coding diseases in the International Classification of Diseases, 10th revision (ICD10), the existence of “other” option for the disease classification in the CSO software, the inability to receiving CSO reports, and the complexity of the CSO software were reported as the main reasons for the complexity of the surveillance system.
|Table 2: Distribution of experts surveillance system's response to the simplicity of CMSS|
Click here to view
With regard to the flexibility domain of the CMSS, the experts were asked some questions about a new list of diseases to the CMSS in such a way that this question showed the flexibility of the CMSS program in terms of cost and time. And according to this question, 52.75% of the experts believed that adding a new list of diseases to the ICD10 grouping or adding new features from the deceased child to the CMSS is affordable in terms of time. Only 17% of the experts opposed the change in the CMSS in terms of time and believed that it would be difficult to make these changes in the CMSS due to time constraints. Also, the results of investigation about making changes in the CMSS from the point of view of workload and the associated problems showed that 48.35% of the CMSS experts believed that it was very easy to make changes in this system.
Also, 46.15% of the CMSS experts believed that adding a new list of diseases to the ICD10 grouping in CMSS is affordable in terms of time [Table 3]. Some discussion was done in the focus group section on the flexibility of the CMSS in terms of the characteristics of time, workload, and cost in various parts of the system. Most of the childcare system experts believed that making changes such as disease encoding, improving CSO software; reporting formats; and the methods of diagnosing the cause of death by doctors, and adding new items to the program would be affordable for the surveillance system program in terms of time, workload, and cost.
|Table 3: Distribution of experts surveillance system's response to the flexibility of CMSS|
Click here to view
Regarding the acceptability domain of the CMSS, the results showed that 44% of experts believed that the surveillance system questionnaire was filled by patients' experts and 44.5% of the experts believed that the deceased child's families did not have the necessary cooperation to complete the questionnaire information [Table 4]. In the focus group, it was referred to as relatives' unwillingness to cooperate in completing the information of the deceased's questionnaire.
|Table 4: Distribution of experts surveillance system's response to the acceptability of CMSS|
Click here to view
| Discussion|| |
In order to improve the timeliness status in surveillance systems, a report should be made of the public health status, the data collection procedure, the reporting monitoring process, and the time intervals for monitoring the surveillance system. In addition, the information system of the surveillance system should pay attention to the timeliness data, since timeliness is one of the most important determinants of decision-making and policy-making in the health system.,, Timeliness in the CMSS was influenced by many factors such as the overall structure and appearance of the program, the time required to submit mortality reports, and the timeliness of feedback and reports. In the meantime, most experts believed that the timeliness (24 hours) for immediate death reporting was very low due to limited possibilities and existing problems. Most experts believed that limited hardware facilities such as computer and software systems were the main problems in delaying the immediate reporting of death. Also, most experts believed that the data obtained from questionnaires B, C, and D could not be completed within one week due to their size. In-hospital and out-of-hospital committees are not held at regular intervals because the committee's main members do not attend sessions, so if in-hospital and out-of-hospital committees are not held, there would be no useful outcome for the surveillance system. Yoo et al., in 2009, evaluated the timeliness of nationally notifiable diseases surveillance system in KORE based on their study results. The proportion of cases reported in time was lower for disease groups with a recommended time limit of 1 day compared with 7 days. Time from disease onset to diagnosis generally contributed most to the delay in reporting, which is consistent with our study results. While in He et al.'s study, which was conducted to assess the Australian Pediatric Surveillance Unit (APSU) from 2000 to 2007 on the basis of the CDC guideline, an average 96% of monthly report cards were returned per annum since 2000. Contrary to the results of this study, the timeliness of APSU was timely and acceptable. Feedback is sent to the users of the APSU in a timely and regular basis. This can be explained by differences in hardware, data quality, and feedback in both systems.
Our study results are consistent with other previous studies on infectious diseases.,, Timeliness has been reduced for most diseases in recent years. Timeliness in the CMSS has been improved in terms of the immediate reporting of infectious diseases as compared with the previous studies.,,, Another study showed that the timeliness of the diseases is low based on the national reporting of the disease surveillance system. The results showed that 28% of doctors reported the delay in completing surveillance system questionnaires, and the main reason for this delay was attributed to the volume of questionnaires, which is consistent with the results of Toprani et al.'s study. The results of these studies are consistent with those of the present study regarding the surveillance system. Overall, based on the results of this study, there is limited timeliness for immediate reporting, holding children's death committees on time, and completion of questionnaires and the intended measures are not carried out in the specified time duration.
Proposed practical solution for the timeliness domain of the CMSS: Based on the results of this study, the deadline for the initial report of death and reporting was very limited due to the lack of feedback from the Ministry of Health to beneficiary universities and the failure to report child deaths in the form of a general report; it is suggested that the existing timeframe is increased and child death reports are available to stakeholders and the general public on an annual basis.
Usually, people involved in the CMSS in Iran have not undergone any special training for this purpose and doing the related activities while providing other health services. As a result, the surveillance system should be simple and capable of achieving its goals at the same time.
According to the results of this study, the majority of experts believed that the overall structure of monthly reporting is complex and difficult. According to the reports and results obtained from this study, the experts of this program believed that the program was highly complex due to structural and executive reasons and factors such as inability to store information in CSO software, complexity of ICD10 classification, contradictory questions in questionnaires, lack of reporting capabilities, novice experts, and complexity of the CSO software were mentioned as the most important problems related to the simplicity of the surveillance system. This result is consistent with that of the current study. Toprani et al., in 2014, evaluated the simplicity of the abortion reporting system in New York City. According to the results of other studies, 95% of the experts believed that reporting was very constructive and useful, while 52% of the experts referred to the complexity of the reporting questionnaires and believed that completing the questionnaires requires training and is a complex process and also stated that the certificate of death and the mentioned causes of death are vague, which is consistent with the result of the present study. Unlike the results of this study, the study by He et al. showed that the Australian childcare system was very simplistic. This can be explained by differences in hardware, the presence of specialized and trained experts in both the systems.
Proposed practical solution for the simplicity domain of the CMSS: reducing the volume and questionnaire questions, reviewing and revising CMSS questionnaires based on the expert opinion of this system, training of care system experts, organizing virtual training courses for experts in the care system, and assessing the extent to which experts are mastered using virtual test to confirm the certificate for care system experts.
If changes occur in the CMSS, it will be possible to implement flexibility at a low cost and effort so that, for instance, it would be possible to add a disease or new individuals' profiles to the previous surveillance system in less time and with little human resources. In the current surveillance system, adding a new list of diseases or new profiles of the deceased child or making changes in the surveillance system implementation procedure should be affordable in terms of time, cost, and practicality of the program. The results of our study are consistent with the results of the study conducted by He et al., which reported that the flexibility of the APSU is high and acceptable. In this study, time, cost, and operationalization of these processes were examined; most experts believed that making changes in the surveillance system was affordable in terms of time. They also believed that these changes could be applied in the surveillance system within a short period of time. A total of 49% of the experts considered these changes to be practical and 37% of them considered the changes to be positive in terms of cost. These results are consistent with the results of the previous study.
The proposed practical solution for the flexibility domain of the CMSS: Make changes to the CMSS if needed.
The acceptability domain refers to the willingness of the deceased child's families to report and also to participate in the CMSS, which will guarantee the success of the surveillance system. Most of the experts believed that the deceased children's families were not able to respond due to their child's death and, in some cases, the deceased's relatives did not cooperate due to the possibility of legal conflict and fear of conviction. Therefore, this program does not enjoy good acceptability among the deceased family, and the experts believed that parents' cooperation status would be improved if parents were given more time to complete the questionnaires and experts were provided with training courses and good counseling principles. Therefore, they will not have a positive effect on reducing the mortality of children. Other studies on the acceptability of the surveillance system such as the infectious  disease surveillance system , and the vaccination surveillance program  showed that this program enjoys high acceptability compared to the CMSS in Iran. This difference may be justified due to the difference in the subject matter.
The proposed practical solution for the acceptability domain of the CMSS: Publication of timely national, regional, and provincial reports (feedback) by the certified medical management safety specialist (CMMSS) and the Ministry of Health experts in the CMSS and publication of annual child deaths by the Ministry of Health for public access.
Study strengths and weaknesses: Our study had a strong point to be noted, the evaluation of the CMSS was done according to the CDC guidelines. To increase the accuracy of the results, CMSS experts were randomly selected from all centers under the care system. Another strength of this study is that data collection was done by questionnaires and holding focus groups. The only major limitation of this study was the subjectivity of the data.
| Conclusions|| |
The findings of this study reveal that CMSS has some limitations and problems in the timeliness, simplicity, and acceptability criteria, which can be resolved according to the executive experts in Iran's medical sciences universities. But this program has an appropriate situation in terms of flexibility.
The authors would like to thank all experts of the CMSS for their contribution to the study.
Financial support and sponsorship
The results described in this paper were part of student thesis. The present study was supported by a grant from Iran University of Medical Sciences.
Conflicts of interest
There are no conflicts of interest.
| References|| |
World Health Organization. Global Technical Strategy for Malaria 2016-2030. World Health Organization; 2015.
World Health Organization. Global Tuberculosis Control: Surveillance, Planning, Financing: WHO report 2008. Vol. 393. World Health Organization; 2008.
UNICEF. The State of the World's Children 2009: Maternal and Newborn Health. Vol. 9. Unicef; 2008.
World Health Organization. World Malaria Report 2015. World Health Organization; 2016.
World Health Organization. WHO Case Definitions of HIV for Surveillance and Revised Clinical Staging and Immunological Classification of HIV-Related Disease in Adults and Children. 2007.
Kohler PK, Marumo E, Jed SL, Mema G, Galagan S, Tapia K, et al
. A national evaluation using standardised patient actors to assess STI services in public sector clinical sentinel surveillance facilities in South Africa. Sex Transm Infect 2017;93:247-52.
Cecil E, Wilkinson S, Bottle A, Esmail A, Vincent C, Aylin PP. National hospital mortality surveillance system: A descriptive analysis. BMJ Qual Saf 2018;27:974-81.
He S, Zurynski YA, Elliott EJ. Evaluation of a national resource to identify and study rare diseases: The Australian paediatric surveillance unit. J Paediatr Child Health 200945:498-504.
Tan HF, Yeh CY, Chang HW, Chang CK, Tseng HF. Private doctors' practices, knowledge, and attitude to reporting of communicable diseases: A national survey in Taiwan. BMC Infect Dis 2009;9:11.
Chaintarli K, Jackson S, Cotter S, O'Donnell J. Evaluation and comparison of the National Tuberculosis (TB) Surveillance System in Ireland before and after the introduction of the Computerised Electronic Reporting System (CIDR). Epidemiol Infect 2018;146:1756-62.
Close M, de l'onusida S, de l'onusida C. Guidelines for using HIV testing technologies in surveillance: Selection, evaluation and implementation. Learning 2009.
Aylin P, Bottle A, Burnett S, Cecil E, Charles KL, Dawson P, et al
. Evaluation of a national surveillance system for mortality alerts: A mixed-methods study. 2018.
Hall HI, Song R, Gerstle JE 3rd
, Lee LM, HIV/AIDS Reporting System Evaluation Group. Assessing the completeness of reporting of human immunodeficiency virus diagnoses in 2002–2003: Capture-recapture methods. Am J Epidemio 2006;164:391-7.
Yoo HS, Park O, Park HK, Lee EG, Jeong EK, Lee JK, et al
. Timeliness of national notifiable diseases surveillance system in Korea: A cross-sectional study. BMC Public Health 2009;9:93.
Klevens RM, Fleming PL, Li J, Gaines CG, Gallagher K, Schwarcz S, et al
. The completeness, validity, and timeliness of AIDS surveillance data. Ann Epidemiol 2001;11:443-9.
Schwarcz SK, Hsu LC, Parisi MK, Katz MH. The impact of the 1993 AIDS case definition on the completeness and timeliness of AIDS surveillance. Aids 1999;13:1109-14.
Toprani A, Madsen A, Das T, Gambatese M, Greene C, Begier E. Evaluating New York city's abortion reporting system: Insights for public health data collection systems. J Public Health Manag Pract 2014;20:392-400.
Flamm RK, Sader HS, Farrell DJ, Jones RN. Summary of the contemporary (2010) ceftaroline activity among USA pathogens: Report from the assessing worldwide antimicrobial resistance evaluation (AWARE) surveillance program. Antimicrob Agents Chemother 2012. doi: 10.1128/AAC.00330-12.
Darabi A, Hocquet D, Dowzicky MJ. Antimicrobial activity against Streptococcus pneumoniae and Haemophilus influenzae collected globally between 2004 and 2008 as part of the tigecycline evaluation and surveillance trial. Diagn Microbiol Infect Dis 2010;67:78-86.
Garrison MW, Mutters R, Dowzicky MJ. In vitro
activity of tigecycline and comparator agents against a global collection of Gram-negative and Gram-positive organisms: Tigecycline evaluation and surveillance trial 2004 to 2007. Diagn Microbiol Infect Dis 2009;65:288-99.
Jajosky RA, Groseclose SL. Evaluation of reporting timeliness of public health surveillance systems for infectious diseases. BMC Public Health 2004;4:29.
Monteiro SA, Takano OA, Waldman EA. Evaluation of the Brazilian surveillance system for adverse events following vaccination. Rev Bras Epidemiol 2011;14:361-71.
Keramarou M, Evans MR. Completeness of infectious disease notification in the United Kingdom: A systematic review. J Infect 2012;64:555-64.
Doyle TJ, Glynn MK, Groseclose SL, Completeness of notifiable infectious disease reporting in the United States: An analytical literature review. Am J Epidemiol 2002;155:866-74.
Overhage JM, Grannis S, McDonald CJ. A comparison of the completeness and timeliness of automated electronic laboratory reporting and spontaneous reporting of notifiable conditions. Am J Public Health 2008;98:344-50.
[Table 1], [Table 2], [Table 3], [Table 4]