Study design and setting
We conducted a cross-sectional observational study using data gathered as part of routine clinical care with the Child Health Improvement through Computer Automation system (CHICA), a clinical decision support system used by five urban, outpatient pediatric clinics in Indianapolis, IN. CHICA was launched in 2004, and has been in continuous use since. CHICA houses data for over 50,000 patients and nearly 350,000 clinical encounters. All clinics are staffed by pediatricians (the majority) or advanced practitioners, referred to as “providers” or “clinicians” herein. These providers receive initial training on CHICA, and have ongoing onsite technical support available. Data and usage patterns are reviewed weekly by the informatics team, and targeted outreach is conducted as needed.
CHICA collects data about patients via two primary means. One is a Pre-Screener Form (PSF), available in English or Spanish, completed by parents or patients in the waiting room. The PSF includes 20 yes/no questions concerning the child’s health and risk factors (e.g., household violence, maternal depression). The questions on the PSF are derived algorithmically [15] based on the child’s age, responses on previous screeners, information from previous clinic visits, and data in the child’s medical record. While initially printed on paper forms, the PSF is now administered on an electronic tablet. [16]
The Provider Worksheet (PWS) is the second source of data in CHICA. The PWS contains up to six reminder prompts for treating clinicians that are also algorithmically-generated based on the responses on the child’s PSF(s), data in the electronic medical record, and age-appropriate care guidelines. Each of the six reminders on the PWS contains checkboxes, with which the providers indicate their responses to the prompt. If the patient indicated pain on the PSF screening, the PWS would prompt the provider to document the level of pain and counsel appropriately (Fig. 1c). In addition to decision support, the PWS offers a method to quickly document normal or abnormal exams across 15 body systems, according to each clinician’s assessment (Fig. 1d).
Patients in the clinics using CHICA are diverse, with significant proportions of African American and Hispanic children. CHICA distributes nearly 1000 PSF forms and generates between 2000 and 3000 patient-specific prompts for the various providers on the PWS each month. The overall patient-response rates on the PSF are greater than 85%. CHICA has been used to study a variety of phenomena, such as improved developmental screening [17], infant television viewing and maternal depression [18], and mental health outcomes of children exposed to violence or depression [19].
Procedure
In this study, we relied on a specific PSF prompt assessing pain that was asked as a routine part of all patient encounters. For patients less than 12 years old, the question was directed to the parent/guardian, reading, “Is [child’s name] having pain today?” (Fig. 1a and b). For patients 12 years and older, with the intention that the child would complete the PSF himself or herself, the question reads, “Are you in pain today?” If the patient reported pain, the physician received the alert in Fig. 1c, with a numeric pain scale reporting option. We recorded the physician responses to the alert and physical exam documented on the PWS at these encounters. We also collected each patient’s pain report from the preceding visit, if available.
We extracted sociodemographic data from CHICA, including race/ethnicity, sex, age, and the payer source at the index encounter. We determined the PSF delivery method—paper versus electronic tablet—based on the date of the visit and the date each clinic transitioned to tablets. Additionally, we classified each visit date by season, using the calendar months as break points (e.g., December through February as winter). We coded age groups based on National Institute of Child Health and Human Development recommendations: 2 to less than 6 years; 6 to less than 12 years; and 12 to less than 20 years of age. We coded race/ethnicity as Black, White, Hispanic, or Other/Unknown. We coded payer source as commercial/private, public (Medicaid/Medicare), and other (self-pay or no insurance).
CHICA records each child’s clinic-measured height and weight. We calculated BMI as weight in kilograms divided by height in meters squared. We determined BMI percentile using Centers for Disease Control and Prevention (CDC) 2000 growth chart parameters. To eliminate biologically implausible values, we discarded data from patient encounters with height, weight, or BMI percentile modified z-score values beyond pre-determined limits by age, following a technique recommended and validated by the CDC. [20]
BMI category was stratified into underweight (less than 5th percentile), normal BMI (5th to less than 85th percentile), overweight (85th to less than 95th percentile), obese (95th to less than 120% of the 95 percentile), and severely obese (120% of the 95th percentile or greater) classes, again following recommendations for stratifying extreme BMI values. [21]
Data collection
Data were gathered from all patient encounters between July 2009 and February 2016. Eligibility criteria for this study were age between 2 and 19 years, inclusive, and having a PSF pain question response, height, and weight at the same visit. Patient encounters were excluded if any covariate data were missing; no data were imputed. For patients with multiple visits in the CHICA system, we used the last clinical encounter that met eligibility criteria (index encounter).
The study protocol was approved by the Indiana University Institutional Review Board, and included a waiver of informed consent because no interventions were being implemented, impracticability, and minimal risk.
Statistical analysis
We tested for association between potential variables using Pearson’s chi-squared, with Cramer’s V to measure effect size. Unadjusted (univariable) logistic regression was used to identify variables associated with patient-reported pain, with a threshold of P < 0.10 to be retained in an adjusted model. Despite not meeting this cutoff, we retained payer source in the adjusted model as a means-tested proxy for income, and in turn, socioeconomic status (SES). [22] We used multivariable logistic regression models to identify factors associated with positive pain reports. Variables included were BMI category, age category, sex, race, payer source, clinic site, PSF delivery method, season, and pain reported on the PSF at the preceding visit. The most common levels of each variable was chosen as the reference level for the models, except for age, where we chose the youngest group as a chronologic baseline. Due to the association between clinic site and race, we also included an interaction term between clinic and race. We chose to use an interaction fixed effect instead of a random effect for clinic due to the limited number of clinics and to describe the clinic-race mechanisms more deeply. Two models were prepared: the first included all patients at their most recent qualifying visit, and the second was a subset limited to patients for which there was a preceding visit with a response to the pain question. Separately, we created a multivariable logistic regression model to determine the odds ratio of reporting pain based on abnormal exam variables. All multivariable regressions were two-sided tests with alpha = 0.05. We used R 3.3.1 [23] to conduct the analyses.