Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

The pot calling the kettle black: the extent and type of errors in a computerized immunization registry and by parent report

  • Shannon E MacDonald1, 2Email author,
  • Donald P Schopflocher2, 3 and
  • Richard P Golonka4
BMC Pediatrics201414:1

DOI: 10.1186/1471-2431-14-1

Received: 2 October 2013

Accepted: 27 December 2013

Published: 4 January 2014

Abstract

Background

Accurate classification of children’s immunization status is essential for clinical care, administration and evaluation of immunization programs, and vaccine program research. Computerized immunization registries have been proposed as a valuable alternative to provider paper records or parent report, but there is a need to better understand the challenges associated with their use. This study assessed the accuracy of immunization status classification in an immunization registry as compared to parent report and determined the number and type of errors occurring in both sources.

Methods

This study was a sub-analysis of a larger study which compared the characteristics of children whose immunizations were up to date (UTD) at two years as compared to those not UTD. Children’s immunization status was initially determined from a population-based immunization registry, and then compared to parent report of immunization status, as reported in a postal survey. Discrepancies between the two sources were adjudicated by review of immunization providers’ hard-copy clinic records. Descriptive analyses included calculating proportions and confidence intervals for errors in classification and reporting of the type and frequency of errors.

Results

Among the 461 survey respondents, there were 60 discrepancies in immunization status. The majority of errors were due to parent report (n = 44), but the registry was not without fault (n = 16). Parents tended to erroneously report their child as UTD, whereas the registry was more likely to wrongly classify children as not UTD. Reasons for registry errors included failure to account for varicella disease history, variable number of doses required due to age at series initiation, and doses administered out of the region.

Conclusions

These results confirm that parent report is often flawed, but also identify that registries are prone to misclassification of immunization status. Immunization program administrators and researchers need to institute measures to identify and reduce misclassification, in order for registries to play an effective role in the control of vaccine-preventable disease.

Keywords

Immunization Vaccination Immunization status Immunization information system (IIS) Registry Parent report Misclassification

Background

In order to optimize the effectiveness and safety of childhood immunization programs it is essential to have an accurate record of children’s immunization status, and ultimately population-level immunization coverage. Such information is essential for clinical care, administration and evaluation of immunization programs and policies, vaccine research, and tracking vaccine-associated adverse events [17].

Accurate assessment of immunization status is dependent on having a valid, comprehensive, and accessible source of data [8]. While hard-copy provider records (typically clinic charts) are a trusted source of immunization status [9, 10], they are not a feasible, cost-effective, or easily accessible method for tracking individual or population level coverage on an ongoing basis [11]. Parent-held records or parent recall are alternatives that are commonly used in immunization research, but these sources are often inaccurate; parent-held records typically underestimate coverage, while parent recall tends to overestimate it [8, 9, 1215]. Population-based electronic immunization registries, also known as Immunization Information Systems (IIS), have been proposed as a valid, cost-effective, and accessible option for assessing immunization status [1, 6, 16]. These registries are centralized electronic repositories for immunization data for a specified geographic location that can consolidate immunization records from multiple providers and settings [1]. They have been promoted by immunization advisory bodies in the USA and Canada [1720], and have been proposed as an alternative source of immunization status verification for the National Immunization Survey conducted annually in the USA [21]. However, it has been recognized that additional validation studies are needed to determine the accuracy of registry data and identify areas for improvement [1, 16, 21, 22].

The purpose of this article is to: (a) report on the accuracy of immunization status classification in an immunization registry as compared to parent report, and (b) identify the frequency and type of errors for both sources, in order to identify areas for system improvement.

Methods

Study design

This study was a sub-analysis of a larger research study conducted in 2009–2010 investigating various factors associated with childhood immunization uptake. The study utilized a postal survey to assess the immunization knowledge and experiences of parents of children whose immunizations were up to date (UTD) at age two, as compared to those who were not UTD. The approval of the Health Research Ethics Board at the University of Alberta and participant informed consent were obtained for the study. The details of the postal survey and the results of the multivariate model of factors influencing uptake are currently under review for publication elsewhere.

Study setting and population

The study population for the postal survey was selected from a regional immunization registry in the city and surrounding rural communities of Edmonton, Alberta (population 1.1 million) in Canada. This registry includes immunization data on all children who were born in the Edmonton zone, as well as those that moved to the zone and accessed public health services. All routine childhood immunizations in this zone are administered by nurses in community-based public health clinics, recorded on a hard-copy clinic record, and entered in the electronic registry onsite by designated clerical staff. Each child is assigned to a ‘home’ public health clinic where their chart is stored. If an appointment is made at a different location, the chart is transferred prior to the visit, any immunizations administered are recorded on the chart and entered into the registry on site, and then the chart is transferred back to the home clinic. The registry software only permits entry of a vaccine dose with a ‘valid date of administration' , i.e. a specific date must be attached to each vaccine dose for it to be entered in the registry.

To be considered UTD, children had to have received all doses of the five vaccines recommended in the Alberta provincial schedule at that time: Diphtheria, Tetanus, acellular Pertussis, Polio and Haemophilus influenzae type b (4 doses); Measles, Mumps, Rubella (1 dose); Varicella (1 dose or history of disease); Meningococcal C conjugate (3 doses); and Pneumococcal 7-valent conjugate (4 doses). This schedule is not consistent across Canada since each province and territory sets its own immunization schedule. The study accounted for variation in the number of doses required for children who were older at initiation of the series or due to individual clinical conditions (e.g. if first dose of meningococcal vaccine is given between 4 and 12 months old, only two doses are required).

Sampling

Children were randomly selected from a one year birth cohort (born May 1, 2006 – April 30, 2007) in the regional immunization registry. An algorithm was used to identify children who were UTD (n = 371) or not UTD (n = 371) at two years of age. The sample size was determined using an effect size and response rate from a previous study [23], a 95% Confidence Interval, 80% Power (ß = 0.20), and a 1:1 ratio of cases to controls.

Postal survey administration

Parents of children identified from the registry were mailed a postcard informing them about the study, followed by the survey a week later. If no response was received after 3 weeks, a reminder postcard was sent, followed by a replacement survey 3 weeks later. In addition to asking about their beliefs and experiences with immunizations, the survey asked parents to report their child’s immunization status according to parent-held records or recall. Specifically, parents were asked whether their child had: (a) not received any immunizations; (b) received some, but not all, the immunizations for their age; or (c) received all the immunizations for their age. Parents were not required to consult the parent-held record due to concerns that excessive participant burden might adversely impact the response rate.

Confirmation of immunization status and data analysis

In cases where there was a discrepancy in immunization status between the registry and parent report, the original hard-copy clinic record was reviewed. The clinic chart was considered the ‘Gold standard’. Given that immunizations are documented on this clinic chart at time of administration, errors in this record are unlikely and no other data source would have superior accuracy. When the parent report of immunization status agreed with that in the registry, no confirmation using clinic charts was undertaken. For this sub-analysis of the larger study, proportions and confidence intervals for errors in reporting were calculated and the type and frequency of errors were determined.

Results

Survey response

Of 1342 potential study subjects, 274 were ineligible due to invalid addresses, 18 withdrew from the study, 589 did not respond, and 461 (331 UTD and 130 not UTD) returned a completed questionnaire. After removing undeliverable surveys from the denominator, the final response rate was 43% (461/1068). Respondents were more likely to be UTD (71.8% UTD, 95% CI: 67.5% - 75.7%, 331/461) as compared to of non-respondents (50.9% UTD, 95% CI: 46.9% - 55.0%, 300/589).

Amount and reasons for error

There were a total of 60 discrepancies between the registry and parent report of immunization status among the 461 survey respondents. Chart review indicated that 44 children were misclassified due to parent reporting and 16 due to registry errors. Table 1 presents the number and direction of misclassification errors from each source. None of the 315 children identified as UTD by the registry were misclassified, while 11.0% (95% CI: 5.9% - 16.0%) of the children recorded as not UTD by the registry (16/146) were misclassified, i.e. they were reported as UTD by parents and confirmed by chart review. The level of error for parent report was the inverse, 11.3% (95% CI: 8.1% - 14.5%) of children reported as UTD by parent report (42/371) were considered not UTD by the registry and confirmed by clinic chart review, while only (2.2%, 95% CI: 0.8% - 5.3%) of the children reported as not UTD by parents were actually UTD (2/90). The frequencies of specific reasons for misclassification from each data source are provided in Table 2.
Table 1

Registry and parent report versus clinic record

  

Clinic record (Gold standard)

 
  

UTD

Not UTD

Total

Registry

UTD

315a

0a

315

Not UTD

16b,c

130a

146

 

Total

331

130

461

Parent report

UTD

329a

42b,c

371

Not UTD

2b,c

88a

90

 

Total

331

130

461

aRegistry and parent report agreed, so no clinic record review conducted.

bConfirmed by clinic record review.

cIndicates misclassification errors.

Table 2

Number and types of misclassification errors

Types of errors in registry

n

 

Dose not entered in registry

1

 

History of varicella disease not entered in registry

4

 

Fewer doses required due to age at first dose, but not noted in registry

2

 

Doses given out of region, but not entered in registry

5

 

Child moved from another province and considered complete by other province’s schedule, but not noted in registry

4

Subtotal

16

Types of errors in parent reporting

n

 

Parent reported child UTD, although:

 
  

Missed dose(s) (unexplained intentional or unintentional misreporting)

25

  

Refused varicella vaccine

6

  

Did not complete immunizations until after received survey

11

 

Parent reported child not UTD, although actually complete (unexplained intentional or unintentional misreporting)

2

Subtotal

 

44

Total number of errors from all sources

60

Discussion

Differential accuracy of reporting and its implications

The results of this study confirm previous findings in other settings that parent reporting of immunization status is not always accurate [8, 9, 1215], but also identifies potential limitations of immunization registry data. The amount of error in the registry found in this study was 3.5%; 16 of 461 immunization records in the final study sample (presuming there were no errors in concordant records that were not checked). This is likely somewhat reassuring to the registry administrators, who consider <3% to be an acceptable amount of error. However, if registries are to be heralded as the most accurate and reliable source for tracking immunization coverage in the future, this level of error is noteworthy when interpreting coverage calculations.

The differential accuracy of reporting children UTD versus not UTD by parent report and the registry is of particular interest (see Table 1). Specifically, we found that parents were generally very accurate in reporting their child as not UTD (only 2.2% were misclassified), but not for reporting them as UTD (11.3% were misclassified). In contrast, the registry was extremely accurate when recording a child as UTD (no errors), but less so when not UTD (11% were misclassified). Only two previous studies have simultaneously compared registry and parent reporting to a third Gold standard [10, 24], and only one [10] described the differential accuracy of UTD/not UTD for the registry versus parent report.

These findings have important implications for program administration, clinical follow-up of individual children, and vaccine research. When evaluating immunization program effectiveness, any child classified as not UTD in a registry may need to be verified by chart review before drawing conclusions about coverage levels for a given region or clinic. In terms of clinical care, a child classified as not UTD in the registry should be verified in the clinic chart before recall/reminder notices are sent to parents. Further validation of immunization through consultation with parents before additional doses of vaccine are administered would avoid administering “extra” vaccine doses, as well as contribute to a self-correcting process to improve registry accuracy. In our study, no children were actually “over-immunized” due to registry error because registry information is always confirmed in the clinic chart for pre-school aged children before vaccines are administered. However, errors could occur after the child turns 5 years, as the chart is then moved into storage and immunization providers utilize the registry for clinical care decisions. In other settings with similar protocols, or where multiple immunization providers access the registry in a clinical setting (i.e. at point of care), it is entirely possible that children could be “over-immunized” based on erroneous registry information. In contrast, when a parent presents their child for medical care and reports their child as not UTD for immunizations, they can be considered generally accurate and appropriate follow-up, including supplementary doses, should be pursued. For research purposes, such as vaccine effectiveness or adverse event studies, our findings suggest that a registry may be the best option for sample selection if the aim is to include both UTD and not UTD children in a study, given the lower overall error rate, as compared to parent report. A registry is also preferable if only UTD children are being studied (no misclassification, compared to parent report), whereas parent report would be a more valid source for identifying subjects if the primary focus is not UTD children.

Reasons for misclassification

Perhaps the most valuable contribution of this study is new knowledge about the specific types of errors contributing to misclassification in immunization registries and parent report (see Table 2). Few studies have reported this information, yet this is a critical element in identifying potential improvements to parent/public health education, quality control, follow-up of incompletely immunized children, and system improvements.

The types of registry errors identified in the limited number of previous studies include: errors in transcription of number or dates of doses administered, and errors in transcribing vaccine formulation, manufacturer, or lot number [25, 26]. Our study is the first to identify failure to transcribe varicella disease history as a considerable source of registry error (accounting for 25% of registry errors). Failure to transcribe factors that reduce the required number of vaccine doses, including late initiation of vaccine series and doses given out of the region, were also a significant source of error (> 40% of registry errors). Registry errors due to children moving from another province and being considered complete by the other province’s schedule (another 25% or registry errors) is a problem unique to the Canadian context, where immunization schedules can vary substantially between jurisdictions. It is noteworthy that the registry used in our study was probably more comprehensive (i.e. fully inclusive of the target population) than in other settings, due to the one-provider system for immunizations in Alberta; thus errors identified in our study might be further compounded in multi-provider settings due to record-scattering [27, 28].

Identifying the reasons for errors in registry data is crucial in determining strategies for improvement. Earlier studies have found that non-transcribed data are sometimes found in locations not routinely transcribed (e.g. discharge summaries, encounter notes) or are in a format not conducive to transcription (e.g. stated as ‘up-to-date' , but no specific dates given) [29]. Our study found similar issues, as well as possibly a lack of awareness by data entry clerks that notations, such as varicella disease history and age at first dose, are relevant data in determining immunization status. These problems with transcription of charted data identified in ours and other studies [25, 26, 29], suggest the need for adequate training of data entry personnel and the need to assess the format of charting forms to facilitate consistent recording of relevant data in the appropriate location for ease and completeness of transcription. Administrators of immunization registries can aid in assuring and improving data accuracy by adopting strategies to decrease the potential for misclassification, including direct electronic data entry [30, 31], electronic data transfer [10, 25], double data entry [26], and audit procedures [16].

The types of errors in parent reporting of immunization status are also of interest. Although the majority of errors (more than 50%) were unexplained, and are likely due to intentional or unintentional misreporting, other types of errors suggest the need for parent education and system improvements. The fact that some parents (14%) who refused varicella vaccine still considered their child to be UTD suggests that these parents did not recognize this vaccine as part of the ‘routine’ immunization series. This is congruent with findings in our larger study, that many parents are selectively refusing the varicella vaccine because they do not consider it an ‘essential’ element of routine childhood immunizations. Another interesting finding was that a number of parents (25% of parent errors) who had not completed their children’s immunizations (more than 6 months after the final dose was due) did so after receiving the survey in the mail. Presumably the survey acted as a reminder mechanism for completion of the series, which speaks to the need for effective follow-up of non-UTD children in this setting and the value of reminder/recall systems.

Strengths and limitations

This study has a number of specific strengths that enable it to contribute new and valuable knowledge regarding the accuracy of immunization registries. This is one of the few studies comparing two alternate sources of data on immunization status to a third Gold standard and is the only published Canadian study to assess the accuracy of an immunization registry. The one-provider system for immunizations in this setting was a particular strength as it virtually eliminated the possibility that ‘record-scattering’ of provider records biased the Gold standard [27, 28]. While the uniqueness of this system might influence the magnitude of data errors (i.e. underestimating what may occur in multi-provider systems), the implications of such reporting errors and the reasons for misclassification that we identified have broad applicability to other regions utilizing immunization registries for surveillance and clinical care.

There were some limitations to this study, which need to be considered in the interpretation of the findings. As study subjects were selected on the basis of immunization status, we were unable to make inferences about prevalence of UTD/not UTD status, and since subjects were not selected on the basis of the Gold standard, reporting of sensitivity, specificity, and predictive values would be misleading. We accept this limitation since a case–control study was the best design to obtain a substantial number of not UTD children in a population with relatively highly immunization coverage [32], and because there is a recognized need to assess data accuracy in immunization program research of all study designs, not merely cohort and cross-sectional studies [1]. The fact that registry accuracy was only assessed for respondents to the survey suggests the potential for bias in this study. Typically survey non-respondents are more likely to be not UTD [10], as was true in our study. Since registry errors were more common for not UTD records, our study likely under-estimates the number of registry errors. Finally, an assumption was made that only incongruent reports of immunization status between parent and registry data need be adjudicated by clinic charts. This assumption is considered justified, since the likelihood of registry and parent errors being in the ‘same direction’ (i.e. both erroneously UTD or not UTD) is minimized due to the fact that (a) the registry software only accepts doses with a specified date, which largely limits mistakes to ‘errors of omission’ [26], and (b) parents who report their child as not UTD are typically accurate [11, 33]. Nonetheless, we recognize that without adjudicating all 461 records, this assumption does leave a source of potential bias.

Conclusions

Despite the significant benefits of population-based immunization registries, our study highlights the potential challenges in ensuring the accuracy of this data source. Clearly, registry records should not always be presumed superior to parent report. At the individual level, parents may be more accurate at identifying their child as not UTD, while the registry is more accurate at identifying UTD children. At the population level, coverage derived from the registry may under-estimate coverage rate, while parent reports tend to over-estimate coverage.

Studies such as this one contribute knowledge needed to improve the quality, completeness, and regional comparability of immunization registries before they can be considered a valid and reliable source of data on immunizations status [21]. We strongly recommend further targeted studies of registry data accuracy in other settings. We also suggest that researchers utilizing immunization registries conduct a quality assessment of their data source, including using appropriate algorithms to confirm classification of immunization status and/or assessing a random sample of subjects in the registry to ascertain the accuracy of the registry versus a Gold standard.

The ultimate goal of any immunization tracking system is to improve the protection of the population from vaccine-preventable disease. As childhood immunization schedules become more complex, parent reporting is likely to become less and less accurate; and as provider records become more and more scattered due to our increasingly mobile society [34], registries have the potential to be not only the best, but the only viable method for tracking individual and population level coverage. This increasing dependence on registries can lead to improvements in population and individual health if appropriate measures are instituted to ensure the accuracy of this data source.

Declarations

Acknowledgements

Christine Newburn-Cook PhD, RN (deceased) made substantial and invaluable contributions to the conception and design of this study. Shannon MacDonald received funding support during her doctoral training from the Canadian Child Health Clinician Scientist Program (CCHCSP), a Canadian Institutes of Health Research (CIHR) Strategic Training Initiative; Women’s and Children’s Health Research Institute; University of Alberta Faculty of Nursing; Alberta Innovates-Health Solutions; and the Izaak Walton Killam Memorial Trust.

Authors’ Affiliations

(1)
Department of Pediatrics, Faculty of Medicine, University of Calgary
(2)
Faculty of Nursing, University of Alberta
(3)
School of Public Health, University of Alberta
(4)
Alberta Health Services, Health Protection, Communicable Disease Control

References

  1. Placzek H, Madoff LC: The use of immunization registry-based data in vaccine effectiveness studies. Vaccine. 2011, 29: 399-411. 10.1016/j.vaccine.2010.11.007.View ArticlePubMedGoogle Scholar
  2. Greene SK, Shi P, Dutta-Linn MM, Shoup JA, Hinrichsen VL, Ray P, et al: Accuracy of data on influenza vaccination status at four vaccine safety datalink sites. Am J Prev Med. 2009, 37: 552-555. 10.1016/j.amepre.2009.08.022.View ArticlePubMedGoogle Scholar
  3. Laroche JA, Diniz AJ: Immunisation registers in Canada: progress made, current situation, and challenges for the future. Euro Surveill. 2012, 17: 17-Google Scholar
  4. Linkins RW, Feikema SM: Immunization registries: the cornerstone of childhood immunization in the 21st century. Pediatr Ann. 1998, 27: 349-354.View ArticlePubMedGoogle Scholar
  5. Wood D, Saarlas KN, Inkelas M, Matyas BT: Immunization registries in the United States: Implications for the practice of public health in a changing health care system. Annu Rev Public Health. 1999, 20: 231-255. 10.1146/annurev.publhealth.20.1.231.View ArticlePubMedGoogle Scholar
  6. Salmon DA, Smith PJ, Navar AM, Pan WK, Omer SB, Singleton JA, et al: Measuring immunization coverage among preschool children: past, present, and future opportunities. Epidemiol Rev. 2006, 28: 27-40. 10.1093/epirev/mxj001.View ArticlePubMedGoogle Scholar
  7. Freeman VA, DeFriese GH: The challenge and potential of childhood immunization registries. Annu Rev Public Health. 2003, 24: 227-246. 10.1146/annurev.publhealth.24.100901.140831.View ArticlePubMedGoogle Scholar
  8. Bentsi-Enchill A, Duclos P, Scott J, MacIsaac K, Halperin S: Childhood pertussis immunization status as reported by parents and the completeness of public-health and physician records in Nova Scotia. Can Commun Dis Rep. 1996, 22: 201-204.PubMedGoogle Scholar
  9. Dorell CG, Jain N, Yankey D: Validity of parent-reported vaccination status for adolescents aged 13–17 years: national immunization survey-teen, 2008. Public Health Rep. 2011, 126 (2): 60-69.PubMedPubMed CentralGoogle Scholar
  10. Stecher DS, Adelman R, Brinkman T, Bulloch B: Accuracy of a state immunization registry in the pediatric emergency department. Pediatr Emerg Care. 2008, 24: 71-74. 10.1097/PEC.0b013e318163db4d.View ArticlePubMedGoogle Scholar
  11. Suarez L, Simpson DM, Smith DR: Errors and correlates in parental recall of child immunizations: effects on vaccination coverage estimates. Pediatrics. 1997, 99: e3-View ArticlePubMedGoogle Scholar
  12. Miles M, Ryman TK, Dietz V, Zell E, Luman ET: Validity of vaccination cards and parental recall to estimate vaccination coverage: a systematic review of the literature. Vaccine. 2013, 31: 1560-1568. 10.1016/j.vaccine.2012.10.089.View ArticlePubMedGoogle Scholar
  13. Lister S, McIntyre PB, Burgess MA, O’Brien ED: Immunisation coverage in Australian children: a systematic review 1990–1998. Commun Dis Intell. 1999, 23: 145-170.PubMedGoogle Scholar
  14. Bolton P, Holt E, Ross A, Hughart N, Guyer B: Estimating vaccination coverage using parental recall, vaccination cards, and medical records. Public Health Rep. 1998, 113: 521-PubMedPubMed CentralGoogle Scholar
  15. Goldstein KP, Kviz FJ, Daum RS: Accuracy of immunization histories provided by adults accompanying preschool children to a pediatric emergency department. JAMA. 1993, 270: 2190-2194. 10.1001/jama.1993.03510180060034.View ArticlePubMedGoogle Scholar
  16. Davidson AJ, Melinkovich P, Beaty BL, Chandramouli V, Hambidge SJ, Phibbs SL, et al: Immunization registry accuracy: improvement with progressive clinical application. Am J Prev Med. 2003, 24: 276-280. 10.1016/S0749-3797(02)00638-4.View ArticlePubMedGoogle Scholar
  17. Public Health Agency of Canada (PHAC: Canadian immunization registry network (CIRN). 2004, Available at http://www.phac-aspc.gc.ca/im/cirn-rcri/ Google Scholar
  18. Federal/Provincial/Territorial Advisory Committee on Population Health and Health Security (ACPHHS): National immunization strategy: final report. 2004, Available at http://www.phac-aspc.gc.ca/publicat/nis-sni-03/ Google Scholar
  19. National Vaccine Advisory Committee (NVAC): Development of community- and state-based immunization registries: report of the national vaccine advisory committee. 1999, Available at http://archive.hhs.gov/nvpo/report071100.pdf Google Scholar
  20. US Department of Health & Human Services: The 2010 national vaccine plan: protecting the Nation’s health through immunization. 2010, Available at http://www.hhs.gov/nvpo/vacc_plan/2010%20Plan/nationalvaccineplan.pdf Google Scholar
  21. Khare M, Piccinino L, Barker LE, Linkins RW: Assessment of immunization registry databases as supplemental sources of data to improve ascertainment of vaccination coverage estimates in the national immunization survey. Arch Pediatr Adolesc Med. 2006, 160: 838-842. 10.1001/archpedi.160.8.838.View ArticlePubMedGoogle Scholar
  22. Kelly JS, Zimmerman LA, Reed K, Enger KS: Immunization information systems national research and evaluation agenda. J Public Health Manag Pract. 2007, 13: 35-38. 10.1097/00124784-200701000-00007.View ArticlePubMedGoogle Scholar
  23. Boulianne N, Deceuninck G, Duval B, et al: Why are some children incompletely vaccinated at the age of 2?. Can J Public Health. 2003, 94: 218-223.PubMedGoogle Scholar
  24. Ortega AN, Andrews SF, Katz SH, Dowshen SA, Curtice WS, Cannon ME, et al: Comparing a computer-based childhood vaccination registry with parental vaccination cards: a population-based study of Delaware children. Clin Pediatr. 1997, 36: 217-221. 10.1177/000992289703600406.View ArticleGoogle Scholar
  25. Mahon BE, Shea KM, Dougherty NN, Loughlin AM: Implications for registry-based vaccine effectiveness studies from an evaluation of an immunization registry: a cross-sectional study. BMC Public Health. 2008, 8: 160-10.1186/1471-2458-8-160.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Samuels RC, Appel L, Reddy SI, Tilson RS: Improving accuracy in a computerized immunization registry. Ambul Pediatr. 2002, 2: 187-192. 10.1367/1539-4409(2002)002<0187:IAIACI>2.0.CO;2.View ArticlePubMedGoogle Scholar
  27. Yusuf H, Adams M, Rodewald L, Lu P, Rosenthal J, Legum SE, et al: Fragmentation of immunization history among providers and parents of children in selected underserved areas. Am J Prev Med. 2002, 23: 106-112. 10.1016/S0749-3797(02)00463-4.View ArticlePubMedGoogle Scholar
  28. Stokley S, Rodewald LE, Maes EF: The impact of record scattering on the measurement of immunization coverage. Pediatrics. 2001, 107: 91-86. 10.1542/peds.107.1.91.View ArticlePubMedGoogle Scholar
  29. Wilton R, Pennisi AJ: Evaluating the accuracy of transcribed computer-stored immunization data. Pediatrics. 1994, 94 (6 Pt 1): 902-906.PubMedGoogle Scholar
  30. Kolasa MS, Chilkatowsky AP, Clarke KR, Lutz JP: How complete are immunization registries? the Philadelphia story. Ambul Pediatr. 2006, 6: 21-24. 10.1016/j.ambp.2005.08.006.View ArticlePubMedGoogle Scholar
  31. Adams WG, Conners WP, Mann AM, Palfrey S: Immunization entry at the point of service improves quality, saves time, and is well-accepted. Pediatrics. 2000, 106: 489-492. 10.1542/peds.106.3.489.View ArticlePubMedGoogle Scholar
  32. Lawrence GL, MacIntyre CR, Hull BP, McIntyre PB: Effectiveness of the linkage of child care and maternity payments to childhood immunisation. Vaccine. 2004, 22: 2345-2350. 10.1016/j.vaccine.2003.10.038.View ArticlePubMedGoogle Scholar
  33. Irving SA, Donahue JG, Shay DK, Ellis-Coyle TL, Belongia EA: Evaluation of self-reported and registry-based influenza vaccination status in a Wisconsin cohort. Vaccine. 2009, 27: 6546-6549. 10.1016/j.vaccine.2009.08.050.View ArticlePubMedGoogle Scholar
  34. Boyd TD, Linkins RW, Mason K, Bulim I, Lemke B: Assessing immunization registry data completeness in Bexar county, Texas. Am J Prev Med. 2002, 22: 184-187. 10.1016/S0749-3797(01)00427-5.View ArticlePubMedGoogle Scholar
  35. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2431/14/1/prepub

Copyright

© MacDonald et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.