General and subspecialist pediatrician perspectives on barriers and strategies for referral: a latent profile analysis
BMC Pediatrics volume 23, Article number: 576 (2023)
Children in need of pediatric subspecialty care may encounter multiple barriers, and multiple strategies have been suggested to improve access. The purpose of this study was to describe the perceptions of general and subspecialty pediatric physicians regarding barriers to subspecialty care and the value of strategies to improve subspecialty access.
We surveyed a national sample of 1680 general pediatricians and pediatric subspecialists in May and June 2020 regarding 11 barriers to subspecialty care and 9 strategies to improve access to subspecialty care, selected from recent literature. Using latent profile analysis, respondents were grouped according to the degree to which they believed each of the barriers impacted access to subspecialty care. Using chi-squared tests, we compared the profiles based on respondent characteristics and perspectives on strategies to improve access.
The response rate was 17%. In 263 responses completed and eligible for inclusion, the barriers most frequently described as “major problems” were wait times (57%), lack of subspecialists (45%) and difficulty scheduling (41%). Respondents were classified into 4 profiles: “Broad concerns,” “Subspecialist availability concerns,” “Clinician communication concerns,” and “Few concerns.” These profiles varied significantly by respondent specialty (p < .001, with medical subspecialists overrepresented in the “Clinician communication” profile, psychiatrists in the “subspecialist availability” profile, and surgeons in the “few concerns” profile); and by respondents’ typical wait time for appointments (p < .001, with physicians with the longest wait times overrepresented in the “subspecialist availability” profile).
We found specific profiles in clinician views regarding barriers to subspecialty care which were associated with perspectives on strategies aimed at overcoming these barriers. These results suggest that health systems aiming to improve subspecialty access should first identify the barriers and preferences specific to local clinicians.
In the United States, on average, one subspecialty visit occurs for each child per year , often at the direction of a primary care provider (PCP). Unfortunately, many referrals result in a significantly delayed subspecialty appointment , an unattended, scheduled appointment, or no scheduled appointment at all [3, 4], while some completed appointments are deemed by subspecialists to be unnecessary [5,6,7]. PCPs [8,9,10], families , and subspecialists [9, 12] alike describe the referral process as ineffective and inefficient, and the breakdown of referrals can threaten patient safety by contributing to delays in diagnoses and treatment . Additionally, patients who do not or cannot access subspecialty care may experience decreased quality of life and increased need for acute care, which may also increase healthcare costs [14, 15].
The complexity of the referral process makes it difficult to assess and improve. Referrals may be impeded by multiple barriers including travel barriers, subspecialist shortages, and communication barriers between referrers, subspecialists, and families [11, 16]. In a 2010 national survey the general pediatrician respondents perceived long wait times, low subspecialist availability, and low acceptance of uninsured patients to be the greatest barriers to subspecialist referral . Less is known, however, about the barriers perceived by subspecialists receiving referrals from pediatricians. In addition, although multiple strategies such as telemedicine, e-consultation [2, 17], the inclusion of generalist pediatricians and/or advance practice providers (APPs) into specialty teams [18,19,20], and referral guidelines  have been designed to improve the referral process, implementation has varied widely [22, 23]. Appropriate implementation of these strategies will depend on identifying the most problematic referral barriers and choosing referral strategies that are well-suited to addressing these barriers  and acceptable to all those involved in referrals, including families, referring clinicians, and subspecialists. To date, very little research has compared the actual or perceived effectiveness of different referral strategies or assessed how strategies might pair with specific situational barriers. We address these knowledge gaps with results from a national sample of pediatricians and pediatric subspecialists, assessing respondents’ views regarding barriers to subspecialty care, referral strategies and the associations between their perspectives on these topics using latent profile analysis.
We analyzed results from a cross-sectional survey administered electronically and by postal mail to a national sample of general pediatricians and pediatric subspecialists, which included multiple topics related to subspecialty referral and telemedicine. Other results from this survey, along with all survey questions, have been reported elsewhere [22, 25]; we focus here on survey questions relating to barriers and potential solutions for subspecialty referral. This survey was administered through postal mail and follow-up email to a random sample of 1680 general pediatricians and medical, surgical, and psychiatric pediatric subspecialists between May and June 2020. Potential respondents were identified from the American Medical Association Masterfile accessed through third-party vendor DMD Marketing Corp (Rosemont, IL), which selected from a list of providers to equally represent general pediatricians and pediatric subspecialists in the four United States census regions. After completing the survey, participants could register for a chance to receive one of thirty-two $100 gift cards. Data from paper surveys were manually entered and double checked by two study members to minimize errors.
Participants were asked to rate 11 potential barriers to subspecialty care as posing a “major problem”, “minor problem,” or “not a problem,” and to rate 9 potential access strategies on a 5-point Likert scale ranging from likely to “significantly worsen” to “significantly improve” access to high-quality subspecialty care. These barriers and strategies were informed by the existing literature on pediatric subspecialty access [3, 10, 24]. Respondents also reported information on their demographics and practice setting. The survey was reviewed by members of the Supporting Pediatric Research on Outcomes and Utilization of Telehealth (SPROUT) collaborative group and was completed through an electronic survey tool (Qualtrics XM, Provo, Utah).
Respondents were eligible to participate if they were general pediatricians, pediatric medical or surgical subspecialists, or child psychiatrists who had completed medical training, provided medical care at least one day per week, and were in the US while completing the survey. We also excluded responses in which the questions assessing barriers to care were not completed. We then utilized Latent Profile Analysis (LPA), a technique which identifies groups (or profiles) of respondents with similarities across multiple variables In this analysis, LPA provided a rigorous methodology with which we could identify groups (or profiles) of respondents based not on one variable but on similar patterns of responses across the 11 survey questions assessing different barriers to subspecialty care (which we reduced to binary variables denoting whether a barrier was perceived to be a “major problem”). The model fit (the degree to which respondents within profiles answered similarly regarding barriers, and to which respondents between profiles answered differently) and classification quality (the frequency with which an individual would be sorted into the appropriate profile) were evaluated for profile solutions with anywhere from 2 to 5 profiles using previously published and empirically supported measures including fit indices, indices of model classification quality, and interpretability [26, 27]. After selecting the best fitting model, we used chi-squared analyses to assess differences between the profiles in terms of clinical features (specialty, years in clinical practice, practice setting, whether respondents’ salaries were primarily determined by relative value units (RVUs), and a typical wait time for nonemergent appointments). We also used chi-squared analysis to assess differences by profile in perceived value of access strategies, with responses reduced into binary variables (perceived worsening or no effect on access vs perceived improvement). LPA was performed using Mplus version 8 (Muthén & Muthén, Los Angeles, California); profile membership was extracted into SPSS (version 23), where chi-square analyses were completed.
Of 1680 surveys distributed, 98 were returned as undeliverable, and we received 301 responses. Because 16 surveys were not eligible for inclusion and 22 respondents did not answer all questions needed for analysis, 263 surveys were analyzed (263/1582 = 17% response rate). These respondents included general pediatricians (n = 116) and pediatric medical (n = 85), psychiatric (n = 42), and surgical (n = 20) subspecialists. The analyzed respondents practiced medicine in 39 different states, and 96% practiced in locations classified as “metropolitan” based on their Rural–Urban Commuting Area codes . As reported elsewhere, respondent characteristics were similar to the population sampled . In the overall population, the barriers most often perceived to be major problems were long wait times (57%), lack of subspecialists (45%) and difficulty scheduling (41%).
The 4-profile solution was chosen as the best fitting model based on a combination of fit indices, classification quality, and interpretability of the profiles (Table 1). We describe these barrier perception profiles as “Broad concerns” (n = 30, 11%) “Subspecialist availability concerns” (n = 96, 36%), “Clinician communication concerns” (n = 23, 9%,), and “Few concerns” (n = 115, 44%). Figure 1 illustrates the perceptions around barriers to subspecialty care for each profile.
The four profiles of respondents varied significantly in their specialties (p < 0.001, Table 2). Medical subspecialists were more frequently categorized as concerned with “Clinician communication” (14%, vs 9% of all respondents) and less frequently categorized as concerned with “Subspecialist availability” (31%, vs 36% of all respondents). Psychiatrists were more frequently categorized as having “Broad concerns” (19%, vs 11% of all respondents) and “Subspecialist availability concerns” (71%, vs 36% of all respondents) profiles, and surgical subspecialists were more likely categorized as “Few concerns” (85%, vs 44% of all respondents). The profiles were not associated with academic practice setting or respondents’ years in practice but were associated with the average time respondents perceived patients waited for a new visit to their clinic, with individuals reporting the longest wait times more frequently falling into “Broad concerns” or “Subspecialist availability concerns” profiles (p < 0.001).
The access strategies felt to be beneficial by the largest number of respondents were telemedicine (85%), referral hotlines (81%), and more subspecialists (80%, Table 3). In contrast, 61% of respondents felt that increasing the number of nurse practitioners and physician assistants would be beneficial to access. Respondents varied significantly by barrier perception profile in their views on telemedicine (p = 0.049) and increasing the numbers of subspecialists (p < 0.001); for both of these strategies, respondents in the “Subspecialist availability concerns” profile were most likely to anticipate a benefit, and respondents in the “Clinician communication concerns” profile were the least likely to anticipate a benefit.
In this national sample of general pediatricians and pediatric subspecialists, respondents reported that the most significant barriers to high-quality subspecialty care were related to availability of subspecialists and subspecialty appointments: long wait times, lack of subspecialists, and difficulty scheduling. Similarly, respondents were most optimistic about strategies aimed at increasing the availability of subspecialists and subspecialty knowledge: telemedicine, referral hotlines, and greater numbers of subspecialists. Despite these general trends, however, latent profile analysis revealed four distinct groups with different concerns. The largest group had relatively “Few concerns” for all the studied barriers. Surgeons were overrepresented in this group, which may relate to other work showing that patients referred to surgical specialists may face less barriers than those referred to medical care . The second largest group expressed concern most frequently for the supply of subspecialists (85%) and the related concern of long wait times (100%). Psychiatrists were overrepresented in this group, consistent with known concerns about the mental health workforce [29, 30]. A smaller group, by contrast, reported low concern for workforce supply and initial access barriers, but higher concern for “Clinician communication” barriers: subspecialists lacking information when first encountering patients (100%) and PCP-subspecialist communication (60%). This group also had the highest level of concern (48%) regarding over-referral. The existence of this “Clinician communication concerns” profile, which was composed primarily of general pediatricians and medical subspecialists, suggests the heterogeneity of barriers to high-quality pediatric subspecialty care, and the importance of defining these barriers and implementing strategies that overcome the specific barriers affecting a given institution, subspecialty, and patient population. This general principle – that referral strategies should be matched with specific clinical settings, should be considered by health systems, health systems researchers, and payers and policymakers interested in improving child access and health outcomes through improved referrals. Since we did not find that respondents in the “Clinician communication concerns” profile were significantly more likely to endorse any of the strategies we proposed, more research may be necessary to identify strategies appropriate for situations where over-referral and gaps in communication are of concern.
This work illustrates two major strengths of latent profile analysis. First, grouping respondents into profiles allowed us to identify heterogeneity among our respondents that extended beyond their answers to any one individual question. We were also able to describe the associations between individuals’ responses to different questions; for example, describing the connections between respondents’ views on barriers and potential solutions, without listing out the associations between views on 9 barriers and 11 referral strategies). However, this study also had limitations: our response rate was only 17%, and although this is a similar rate to other surveys of physicians [31, 32] and the characteristics of the respondents did not differ greatly from the national random sample in measured characteristics (respondent specialty, gender, years in practice, and census region) , they may have differed in some unmeasured characteristics which would have influenced their responses. We also chose to analyze responses from generalists and subspecialists together. This allowed us to note differences in concerns between referral placers and receivers, and to identify barrier concern profiles that included both groups, but may decrease the sensitivity of our results since, for example, subspecialists answered questions about referral barriers only within their own specialty, but generalists were likely considering referrals overall. The third major participant in subspecialty referral, patients and their families, were not included in this survey. Because we asked only about specific access strategies and did not solicit free-response perspectives on access strategies, our conclusions are constrained to the barriers and solutions about which we explicitly asked. We cannot address, for example, the inclusion of pediatric generalists into subspecialty teams. Finally, this study was carried out within the first six months of the coronavirus-2019 pandemic in the United States. This pandemic affected clinical practice in multiple ways, including increasing telemedicine use and decreasing demand for many pediatric subspecialties. These changes may have influenced our survey responses, and it remains to be seen to what degree these pandemic-related shifts will persist.
We found that although many pediatric generalists and subspecialists expressed concerns related to patients’ ability to access subspecialists, this concern was not universal, with some physicians alternatively concerned with over-referral, PCP-subspecialist communication, and information-sharing. Systems hoping to improve access to subspecialty pediatric care should avoid a one-size-fits-all approach and should instead match access strategies to the barriers experienced in specific clinical settings and by specific patient populations.
Availability of data and materials
The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.
Akaike Information Criteria
Bootstrap Likelihood Ratio Test
Latent profile analysis
Primary care provider
Relative value unit
- SSA BIC:
Sample size adjusted Bayesian Information Criteria
Vernacchio L, Muto JM, Young G, Risko W. Ambulatory subspecialty visits in a large pediatric primary care network. Health Serv Res. 2012;47(4):1755–69.
Lai L, Liddy C, Keely E, Afkham A, Kurzawa J, Abdeen N, et al. The impact of electronic consultation on a Canadian tertiary care pediatric specialty referral system: A prospective single-center observational study. PLoS One. 2018;13(1):e0190247.
Bohnhoff JC, Taormina JM, Ferrante L, Wolfson D, Ray KN. Unscheduled Referrals and Unattended Appointments After Pediatric Subspecialty Referral. Pediatrics. 2019;144(6). Available from: https://www.ncbi.nlm.nih.gov/pubmed/31704770.
Zuckerman KE, Cai X, Perrin JM, Donelan K. Incomplete specialty referral among children in community health centers. J Pediatr. 2011;158(1):24–30.
Shah-Becker S, Carr MM. Current management and referral patterns of pediatricians for acute otitis media. Int J Pediatr Otorhinolaryngol. 2018;113:19–21.
Reeder BM, Lyne ED, Patel DR, Cucos DR. Referral patterns to a pediatric orthopedic clinic: implications for education and practice. Pediatrics. 2004;113(3 Pt 1):e163-7.
Shinkunas LA, Klipowicz CJ, Carlisle EM. Inappropriate referrals in pediatric surgery. J Pediatr Surg. 2020. Available from: https://www.ncbi.nlm.nih.gov/pubmed/32660777.
Mehrotra A, Forrest CB, Lin CY. Dropping the baton: specialty referrals in the United States. Milbank Q. 2011;89(1):39–68.
Gandhi TK, Sittig DF, Franklin M, Sussman AJ, Fairchild DG, Bates DW. Communication breakdown in the outpatient referral process. J Gen Intern Med. 2000;15(9):626–31.
Pletcher BA, Rimsza ME, Cull WL, Shipman SA, Shugerman RP, O’Connor KG. Primary care pediatricians’ satisfaction with subspecialty care, perceived supply, and barriers to care. J Pediatr. 2010;156(6):1011-1015 e1.
Ray KN, Ashcraft LE, Kahn JM, Mehrotra A, Miller E. Family Perspectives on High-Quality Pediatric Subspecialty Referrals. Acad Pediatr. 2016;16(6):594–600.
Stille CJ, Primack WA, Savageau JA. Generalist-subspecialist communication for children with chronic conditions: a regional physician survey. Pediatrics. 2003;112(6 Pt 1):1314–20.
Closing the Loop: A Guide to Safer Ambulatory Referrals in the EHR Era. Cambridge, MA: Institution for Healthcare Improvement; 2017. Available from: http://www.ihi.org/.
Schatz M, Cook EF, Nakahiro R, Petitti D. Inhaled corticosteroids and allergy specialty care reduce emergency hospital use for asthma. J Allergy Clin Immunol. 2003;111(3):503–8.
Schatz M, Zeiger RS, Mosen D, Apter AJ, Vollmer WM, Stibolt TB, et al. Improved asthma outcomes from allergy specialist care: a population-based cross-sectional analysis. J Allergy Clin Immunol. 2005;116(6):1307–13.
Zuckerman KE, Perrin JM, Hobrecker K, Donelan K. Barriers to specialty care and specialty referral completion in the community health center setting. J Pediatr. 2013;162(2):409-14 e1.
Rea CJ, Samuels RC, Shah S, Rosen M, Toomey SL. Electronic Consultation: Latest Evidence Regarding the Impact on Referral Patterns, Patient Experience, Cost, and Quality. Acad Pediatr. 2020;20(7):891–2.
Di Guglielmo MD, Plesnick J, Greenspan JS, Sharif I. A new model to decrease time-to-appointment wait for gastroenterology evaluation. Pediatrics. 2013;131(5):e1632-8.
Harrison M, Jones P, Sharif I, Di Guglielmo MD. General Pediatrician-Staffed Behavioral/Developmental Access Clinic Decreases Time to Evaluation of Early Childhood Developmental Disorders. J Dev Behav Pediatr JDBP. 2017;38(6):353–7.
Roberts MD, Christiansen A, O’Hagan B, Jansen E, Augustyn M. Developmentally-Trained Primary Care Clinicians: A Pipeline to Improved Access? J Dev Behav Pediatr JDBP. 2023;44(5):e350–7.
Cornell E, Chandhok L, Rubin K. Implementation of referral guidelines at the interface between pediatric primary and subspecialty care. Heal Amst. 2015;3(2):74–9.
Ray KN, Bohnhoff JC, Schweiberger K, Sequeira GM, Hanmer J, Kahn JM. Use of telemedicine for initial outpatient subspecialist consultative visit: A national survey of general pediatricians and pediatric subspecialists. Heal Amst. 2022;10(1):100600.
Liddy C, Drosinis P, Keely E. Electronic consultation systems: worldwide prevalence and their impact on patient care-a systematic review. Fam Pr. 2016;33(3):274–85.
Ray KN, Kahn JM. Connected Subspecialty Care: Applying Telehealth Strategies to Specific Referral Barriers. Acad Pediatr. 2020;20(1):16–22.
Schweiberger K, Bohnhoff J, Hanmer J, Ray KN. Perceived Usefulness of Increased Telemedicine Use by Pediatric Subspecialists: A National Survey. Telemed J E Health. 2022;28(9):1367–73.
Spurk D, Hirschi A, Wang M, Valero D, Kauffeld S. Latent profile analysis: A review and “how to” guide of its application within vocational behavior research. J Vocat Behav. 2020;1(120):103445.
Nylund KL, Asparouhov T, Muthén BO. Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study. Struct Equ Model Multidiscip J. 2007;14(4):535–69.
Cromartie J. 2023. Cited 2023 Oct 2. Rural-Urban Commuting Area Codes. Available from: https://www.ers.usda.gov/data-products/rural-urban-commuting-area-codes.aspx. Accessed 2 Oct 2023.
Severe Shortage of Child and Adolescent Psychiatrists Illustrated in AACAP Workforce Maps. American Academy of Child and Adolescent Psychiatry; 2022. Cited 2023 Oct 4. Available from: https://www.aacap.org/AACAP/zLatest_News/Severe_Shortage_Child_Adolescent_Psychiatrists_Illustrated_AACAP_Workforce_Maps.aspx. Accessed 4 Oct 2023.
The Future Pediatric Subspecialty Physician Workforce: Meeting the Needs of Infants, Children, and Adolescents. Washington, DC: National Academies of Sciences, Engineering, and Medicine; 2023. Available from: https://doi.org/10.17226/27207.
Cook DA, Wittich CM, Daniels WL, West CP, Harris AM, Beebe TJ. Incentive and Reminder Strategies to Improve Response Rate for Internet-Based Physician Surveys: A Randomized Experiment. J Med Internet Res. 2016;18(9):e244.
Barnhart BJ, Reddy SG, Arnold GK. Remind Me Again: Physician Response to Web Surveys: The Effect of Email Reminders Across 11 Opinion Survey Efforts at the American Board of Internal Medicine from 2017 to 2019. Eval Health Prof. 2021;44(3):245–59.
Supported in part by grants from the NICHD (K23HD088642, Dr. Ray), NCATS (1K12TR004384, Dr. Bohnhoff) and HRSA (T32HP22240, Drs. Bohnhoff & Schweiberger). This project was also supported in part by NIH/NCATS SPROUT-CTSA Collaborative Telehealth Network Grant Number U01TR002626. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
Ethics approval and consent to participate
This study was approved by the Institutional Review Board of the University of Pittsburgh. We confirm that all our methods were conducted in agreement with the Declaration of Helsinki. Participants provided their informed consent by reading an introductory script which reviewed risks, benefits, and voluntary nature of participation before choosing to complete the survey or not. Documentation of consent was waived by the Institutional Review Board of the University of Pittsburgh (approval ID: STUDY19050048) since this would be the only record linking subjects to participation.
Consent for publication
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bohnhoff, J.C., Guyon-Harris, K., Schweiberger, K. et al. General and subspecialist pediatrician perspectives on barriers and strategies for referral: a latent profile analysis. BMC Pediatr 23, 576 (2023). https://doi.org/10.1186/s12887-023-04400-8