- Research article
- Open Access
- Open Peer Review
Development and psychometric properties of the parent version of the Profile of Neuropsychiatric Symptoms (PONS) in children and adolescents
BMC Pediatrics volume 15, Article number: 62 (2015)
The use of neuropsychiatric Patient Centred Outcome Measures (PCOMs) in routine child mental health and paediatric services is very time consuming and often requires multiple scales being completed as no single scale covers all areas of psychopathology. The use of a web-based programme can overcome these problems and contribute to improved use of PCOMs in clinical practice. We aim to develop a web-based scale (using HealthTracker™) to screen and identify young people with significant neuropsychiatric symptoms to enable early intervention.
Qualitative development of the Profile of Neuropsychiatric Symptoms (PONS) and quantitative evaluation of the psychometric properties of the PONS scale (parent version). Parents of 929 from the general population and 147 with neuropsychiatric disorders (5–18 years old) completed the PONS online. In addition, those children with neuropsychiatric disorders were assessed for the presence of current and lifetime psychiatric disorders using the Development and Well-Being Assessment (DAWBA).
The PONS scale (parent version) consists of 30 symptom domains rated on a 7-point scale for both frequency and impairment. We found an intra-class correlation coefficient for single measures was 0.44 (0.42-0.46 95 % CI, F = 22.84, p ≤ 0.0001) and for average measures was 0.96 (0.95-0.96 95 % CI, F = 22.84, p ≤ 0.0001). The factor analysis showed a 4-factor model: Neurodevelopmental Disability; Behavioural and Emotional Dysregulation; Psychoses and Personality Dysfunction; and Anxiety and Depression. The receiver operating characteristic area for the 4-factors was 0.96 (SE = 0.006; 0.95-0.97 95 % CI).
The PONS scale (parent version) is a web-based PCOM on the HealthTracker™ system that is a rapid, engaging measure that has excellent reliability and validity. The system allows for automated scoring and immediate feedback of statistical cut-off points and assists clinicians with diagnostic decision-making and optimises use of clinician time.
Epidemiological research has shown that between 12 % and 14 % of young people have a mental disorder causing significant functional impairment [1, 2], 5 % have neurodevelopmental disorders . However, only about one quarter of them are in contact with specialist mental health services [4, 5].
Methods for assessing the prevalence and community burden of psychiatric disorders in children and adolescents have improved dramatically in the past decade . There is a broad range of standardized structured interviews and self-report assessments that generate Diagnostic and Statistical Manual (DSM) Mental Disorder diagnoses with good reliability and validity . Using neuropsychiatric Patient Centred Outcome Measures (PCOMs) in routine child mental health and paediatric services is very time consuming and often requires multiple scales being completed as no single scale covers all areas of psychopathology. Most children and adolescents attending mental health or neurodisability services have symptoms of more than one emotional or behavioural developmental disorder. Currently, there are many disorder-specific rating scales, but none that are child or parent rated that include symptoms of neurodevelopmental disorders such as attention deficit hyperactivity disorder and autism spectrum disorders alongside symptoms of psychoses, bipolar disorder, anxiety and depression. The commonly used measures such as the Child Behaviour Check List (CBCL) , and the Strength and Difficulties Questionnaire (SDQ)  currently do not cover all the above areas adequately. There is an urgent need to develop brief PCOMs for childhood neuropsychiatric disorders and neurodisability, that can be completed in around 10 min, capture frequency and impairment arising from symptoms of multiple, relevant, co-occurring disorders in a dimensional manner allowing its use across conditions. It would be useful if the PCOM could be used longitudinally to capture change. There is a need to capture domain-specific, dimensional frequency and impairment ratings, which would uniquely allow the PCOM to be used across diagnostic categories, allowing for use in research using the NIH Research Domain Criteria (RDoC)  (http://www.nimh.nih.gov/research-priorities/rdoc/index.shtml). The RDoC project intends to develop, for research purposes, new ways of classifying mental disorders based on behavioral dimensions and neurobiological measures . RDoC attempts to bring the power of modern research approaches in genetics, neuroscience, and behavioral science to the problems of mental illness, studied independently from the classification systems by which patients are currently grouped.
PCOMs include the self/proxy assessments of symptoms, functional status, or other concerns such as patient needs and satisfaction with care ([9–15]. The use of a web-based programme can contribute to improved use of PCOMs in clinical practice . Technology can enhance the lives of most individuals, especially those with neurodisabilities [17–20], who will benefit through the use of the HealthTracker™, an existing web-based health-monitoring platform. HealthTracker™ is an established platform for online collection and storage of medical data that allows multi-modal presentation of questionnaires (including animated scale presentations for young children) and assists in automatically allocating questionnaires based on developmental level rather than chronological age. Online display allows questionnaires to be presented in a client-friendly manner in many languages, assisted by audio recordings for those who have dyslexia and visual impairment [21, 22]. In addition, being web-based, it allows for more frequent assessment without clinic visits.
Children and adolescents with intellectual disabilities and challenging behaviours can make conventional assessment difficult . For the reasons stated above, more work is needed to improve PCOMs, especially harnessing technology, to screen young people and identify early childhood onset of psychiatric disorders to enable early intervention; especially in those with neurodisability. The Profile Of Neuropsychiatric Symptoms (PONS) was developed to screen and capture information on specific symptom domains that are commonly seen in neurodevelopmental, neuropsychiatric and emotional and behavioural disorders in children and young people, developed in a manner to capture change in symptom frequency and impairment across time, using current FDA protocols for development of PCOMs and the NIH RDoC agenda. It was developed with a view to capture symptoms of developmental disorders such as ADHD, ASD, OCD, motor coordination disorder; disruptive disorders such as ODD and Conduct Disorder; Psychoses, Bipolar Disorder and Emerging Personality Disorder; and Anxiety and Depressive Disorders. Currently no existing single scale is able to capture all these symptoms in a scale that can be completed in around 10–12 min, which can also be used to track change between clinic visits.
The objective of this project was to develop the web-based PONS. For this purpose, we followed the FDA recommendations for patient reported outcome measures (currently called PCOMs) [24, 25] and principles used in the Patient Reported Outcomes Measurement Information System (PROMIS). PROMIS is an NIH-funded initiative to develop instruments to be used across chronic conditions as a system of highly reliable, precise measures of patient–reported health status for physical, mental, and social well–being (www.nihpromis.org).
The Profile of Neuropsychiatric Symptoms (PONS) consists of three child versions - for young children (5–7 years old), children (8–11 years old) and adolescents (12–18 years old); a parent or carer version, a teacher version, and a clinician version. This paper will report on the parent version of the PONS and its qualitative development, and the quantitative evaluation of its psychometric properties in child and adolescent controls from the general population and a group with neuropsychiatric disorders.
Phase 1: qualitative development of the PONS scale
Institutional Review Board approval was obtained from the Guy’s and St Thomas’s Hospitals NHS Trust before subject enrolment. Online informed consent was obtained from all control participants (i.e., child, adolescent, parent) before study instruments were administered. In the clinic sample, the PONS was completed as part of routine clinical care and we had consent for anonymized use of all clinical data.
The uniqueness of this instrument is that each domain describes what a child with a specific symptom domain would look like, rather than focusing on individual items, for example, hyperactivity is described as ‘a child who is unable to sit still, is constantly on the go, fidgets, is restless.’ This allows the parent and child to understand what constitutes hyperactivity rather than having separate symptom items to capture the domain of hyperactivity. Initial symptom domains of the PONS scale was based on a detailed literature review, followed by a consensus of child and adolescent psychiatrists and neurodevelopmental paediatricians with extensive experience in neuropsychiatric disorders of young people, and families with children with neuropsychiatric problems. The initial draft of the PONS scale was discussed with a panel of experts of child psychiatrists, paediatricians, neurodevelopmental paediatricians, psychologists, and occupational therapists - each of whom had a minimum of 5 years experience working with children in paediatric or child mental health settings. Each of them was asked to rate the importance and relevance of the various domains for the purpose of the PONS. There was 100 percent agreement regarding the merits of inclusion of the domain in the draft version of the PONS and a consensus reached about its relevance. Redundant or overlapping symptom domains were removed. The second draft of the PONS scale was then presented to young people (aged 5 to 18 years), and their parents for feedback, and to obtain information about comprehensibility and appropriateness of both the content and format of the questions. Focus groups for young people and their parents were conducted separately to understand their different perspectives. The mean duration of these focus groups was 90 min for young people and their parents. The major topics evaluated in the focus groups were structure (e.g., format of the questions, response options), content (e.g., meaning of each symptom domain, language used to describe it), and other aspects such as frequency of administration and recall period. Finally, a discussion between experts in child and adolescent neuropsychiatry and paediatric neurodisability was held based on the summary reports by the focus groups and the final version of the PONS scale was developed for young people and parents, as well as for clinicians. It was ensured that the PONS covered child or parent rated symptoms of neurodevelopmental disorders such as attention deficit hyperactivity disorder and autism spectrum disorders, alongside symptoms of psychoses, bipolar disorder, anxiety and depression. The final PONS scale (parent version) consisted of 30 domains with response options on a 7-point Likert scale because developmentally up to 7-point options can be easily distinguished by adults [26, 27] and feedback from focus groups suggested it to be best option for the PONS. Also, having 7-point Likert scale allows a greater likelihood of being able to capture subtle change accurately across time. The PONS scales are incorporated into the HealthTracker™ platform  and are used in different clinical and research settings. Currently, the HealthTracker™ system is used in two EU FP7 projects - the Suicidality: Treatment Occurring in Paediatrics (STOP study; www.stop-study.com)  and the Managing the Link and Strengthening Transition from Child to Adult Mental Health Care (MILESTONE) project (www.milestone-transitionstudy.eu).
Phase 2: psychometric evaluation of the PONS scale
Subjects and procedures
After Institutional Review Board approval we used two samples in order to examine the psychometric properties of the PONS scale: a) a convenience sample of 929 children and adolescents from the general population (466 boys and 463 girls); children (5–12 years old; mean age 10.1 years, SD = 3.99) and adolescents (12–18 years old; mean age 13.9 years, SD = 3.65). Professional online marketing resources helped identify ideal user groups to target (such as parenting websites). Online guidance was clear that our target audience was parents of primary and secondary school children without a previous history of behavioural and emotional problems, or involvement with child mental health services.
b) 147 children and adolescents (109 boys, 38 girls), aged 5 to 18 years (mean = 11.09, SD = 3.2) with neuropsychiatric disorders, recruited from the Centre for Interventional Paediatric Psychopharmacology (CIPP) at the Maudsley Hospital, London. Parents were interviewed at intake about the neuropsychiatric symptoms of their children using the PONS (parent version). In addition, for the neuropsychiatric condition sample, children and parents were also interviewed for the presence of current and lifetime psychiatric disorders and completed the Development and Well-Being Assessment (DAWBA) .
The data capture, storage, and e-monitoring was conducted online using the HealthTracker™ system. HealthTracker™ is an established health-monitoring platform for online collection, storage of medical data, and data-export to an SPSS database.
SPSS version 20.0 was used for the analyses. Descriptive statistics was used to characterize both samples. Reliability was assessed using Cronbach’s alpha, alpha if deleted analysis, intra-class correlation, and factor analysis using the general population. Exploratory factor analysis with Promax rotation and Kaiser normalization was done with rotation converged in 10 interactions. Principal axis factoring without fixed number of factors was used. Promax rotation was used with a maximum interaction for convergence of 0.25 (Kappa = 4). A priori threshold to determine the loading of factors was set. Given the explorative purpose of the present study and our sample size we set the factors’ loading threshold at >0.25 [28–33]. Only the greater value was considered when cross-loadings presented a gap >0.2 between the loadings. In addition, Kaiser-Meyer-Olikin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity was provided.
Validity was assessed using the receiver operating characteristic (ROC) analyses. This was computed to evaluate the discriminative power of the factors between both samples. This has been detailed in Additional file 1 (details available in the Results section). In addition, independent samples t-tests were performed with grouping variable DAWBA diagnoses (coded in 1 for positive and 0 for negative diagnosis) and single factor’s scores as dependent variables to assess the PONS ability to distinguish between different groups of patients and to classify them on the basis of DAWBA diagnosis.
PONS’ cut off scores, sensitivity, and specificity was also computed. All p-values are for two-tailed tests with α = 0.05.
Phase 1: qualitative development of the PONS scale
Parents specifically supported the strategy of the PONS consisting of domains that describe specific areas of dysfunction and believed that the traditional approach of having many separate items for each domain would lead to a longer scale and reduce uptake. The final PONS scale (parent) consisted of 30 symptom domains. Each domain includes its name domain, its description, and two questions, one about the frequency and the other one about impairment. The HealthTracker™ system ensures that the impairment question only appears if the frequency question is answered as being present, thus speeding up completions. This feature was strongly supported by all the users during the focus groups. All symptom domains are rated on a 7-point scale and the recall period was 1 month for baseline administration.
Phase 2: psychometric evaluation of the PONS scale
The sample with neuropsychiatric conditions consisted of 147 children and adolescents with various psychiatric disorders, some of who had multiple comorbid disorders. 75.5 % had Attention Deficit Hyperactivity Disorder (ADHD) (n = 111), 65.3 % had Autism Spectrum Disorder (ASD) (n = 96), 40.1 % had Oppositional Defiant Disorder (ODD) or Conduct Disorder (n = 59), 21.1 % had psychoses or Bipolar Disorder (BP) (n = 31), 54.4 % had Anxiety or Depressive Disorder (n = 80), 25.9 % had Developmental Coordination Disorder (n = 38), and 24.4 % had Obsessive Compulsive Disorder and/or tics (n = 36).
As shown in Table 1, factor analysis was completed on the 30 symptom domains. A 4-factor model was determined to best fit the data based on the screen plot. Based on the pattern of symptom domain loading, the 4-factors were named: (1) Neurodevelopmental Disability (predominantly ADHD, ASD), (2) Behavioural and Emotional Dysregulation (ODD, CD), (3) Psychoses and Personality Dysfunction (Psychoses, BP, emerging PD, spontaneous abnormal movements ), (4) Anxiety and Depression (Anxiety and Depressive Disorders). These factors capture the bulk of children and adolescents with mental health problems and intuitively follow clinical and diagnostic clusters.
The KMO was 0.97 (X2 = 20,507, 54) with a Bartlett’s test of sphericity of 378, p ≤ 0.001.
Cronbach’s alpha was 0.96 for the 30 PONS symptom domains (referred to as items here). There were no negative inter-item correlations and each item-total correlations were above the threshold of >0.20. In addition, the alpha if deleted analysis showed that the Cronbach’s alpha value reduced if any of the PONS items were dropped, meaning that there was no need to drop any items.
Intra-class correlation coefficient for single measures was 0.44 (0.42-0.46 95 % CI, F = 22.84, p ≤ 0.001) and for average measures was 0.96 (0.95-0.96 95 % CI, F = 22.84, p ≤ 0.001).
Receiver Operating Characteristic (ROC) analysis
The results showed that the Neurodevelopmental Disability score was significantly higher for patients who presented with an ASD and/or ADHD diagnoses (t = 3.999, p < .001). The ODD and/or CD patients had a statistically significant difference in the Behavioural and Emotional Dysregulation score (t = 3.352, p = .001). Moreover, the Psychoses and Personality Dysfunction score was significantly greater for those individuals who had psychoses and/or bipolar disturbance (t = 2.384, p = .0022). Finally, patients with depression and/or anxiety had significantly greater Anxiety and Depression score (t = 3.045, p = .003). Taken together, these results suggest that the PONS can be used as a guide for clinical categorization.
The PONS scale (parent version) is a web-based PCOM that consists of 30 symptom domains rated on a 7-point scale for both frequency and impairment and takes around 10 min to complete online. The PONS ensures that child and parent rated symptoms of neurodevelopmental disorders such as attention deficit hyperactivity disorder and autism spectrum disorders alongside symptoms of psychoses; bipolar disorder, emerging personality disorders; anxiety and depression exist in a single scale which is brief and easy to use. Results show that the PONS scale is a reliable and valid instrument in screening for psychiatric disorders in children and adolescents with neuropsychiatric conditions.
Specifically, we found an internal consistency value very close to 1.0 suggesting excellent reliability and providing evidence that the symptom domains measure a common underlying construct. In addition, we found medium correlation for individual symptom domains and high correlations for the full instrument, suggesting that each symptom domain is unique.
The PONS factor structure matches the domain framework of the PROMIS (www.nihpromis.org). The 4 PONS factors, (1) Neurodevelopmental Disability, (2) Behavioural and Emotional Dysregulation (3) Psychoses and Personality Dysfunction, and (4) Anxiety and Depression align to an extent to the PROMIS Peer Relationship domain, the PROMIS Anger domain, and the PROMIS Paediatric Anxiety and Depression domains. The PROMIS currently does not have an equivalent for Psychoses and Personality Dysfunction in children and does not capture all symptoms (www.nihpromis.org). The PONS uniquely captures symptoms of developmental disorders such as ADHD, Autism Spectrum Disorder, Obsessive Compulsive Disorder, motor coordination disorder, disruptive disorders such as ODD and Conduct Disorder; Psychoses, Bipolar Disorder, and Emerging Personality Disorder; and Depression and Anxiety Disorders. Currently no existing single scale is able to capture all these symptoms in around 10 min. Whilst the Strengths and Difficulties Questionnaires (SDQ) , is a brief, valid and widely used questionnaire, it does not cover psychoses, personality dysfunction, bipolar disorder, etc., and has not been used regularly to capture change between clinic appointments. The Child Behavior Checklist (CBCL)  is much longer, more time consuming and is not a simple PCOM that can be completed in around 10 min and does not have the online optimization available for the PONS using audio recordings or animations for the child versions of the PONS. The PONS is unique in that this is an instrument that has been clearly validated for web-based use. The capture of domain-specific, dimensional frequency and impairment ratings, uniquely allows the PONS to be used across diagnostic categories, allowing for use in research using the NIH Research Domain Criteria (RDoC) . Most importantly, the parents were very pleased with the format of the PONS and it is a PCOM that has been developed with the full input of the users.
Optimized, web-based PCOMs using intelligent branching, audio-assistance for those with reading difficulties, adaptations for visual impairment (with large font-size adaptations), availability in multiple languages increases the likelihood of the PCOM being used. All these have been specifically achieved with the PONS on the HealthTracker™. It is likely to increase the use and completions of questionnaires, improve access to care in geographically isolated populations, could be used for automated triaging, and automated, real-time scoring allows for the results to be available for use by clinicians during clinic visits, even if the PONS was completed in the waiting area whilst waiting to see the clinician.
Feedback from parents was very positive from the focus group. Intelligent branching through online delivery from the HealthTracker™ system minimises completion time (maximum 12 min). The HealthTracker™ system also automatically randomises the order of the symptom domains being presented, and thus significantly reduces practice effects.
The HealthTracker™ system has been specifically tested for user acceptance and optimised for the use of children, adolescents and parents, allowing them to complete it effortlessly. The scale also can be read by a recorded voice, which is especially helpful when it comes to those with dyslexia. In addition, the allocation of the versions of the PONS is automatically assigned based on developmental age, rather than chronological age, which is of utmost importance for those with neurodisability.
Furthermore, the HealthTracker™ system allows the PONS to be used for neuropsychiatric screening of children and adolescents and automated triaging in busy clinics. Clinically, the HealthTracker™ platform allows the PONS scores to be automatically calculated and presented in a graphical manner. This real-time shared feedback allows optimal use of face-to-face and non face-to-face clinical time.
The potential limitations of this study include that PONS was primarily developed as a screening measure and to assist in triaging. We have subsequently used the instrument in epidemiological studies and tested its usefulness against other measures. As this instrument is for online use, it was important that normative data be acquired online. This means, however, that we were not able to practically supervise every online entry, and cannot be entirely sure that completions for the normative sample were by parents with children that had no psychiatric problems. We believe that the large numbers recruited means that such errors will be small, and the advantages of test administration in its intended environment outweigh any disadvantages. In addition, the clinical sample is modest in size and comes from a national specialist child and adolescent mental health service (CAMHS) setting and may not be representative of symptom profiles in less severely ill subjects seen in community CAMHS.
However, this neuropsychiatric sample shows that the PONS screened and identified psychiatric conditions with excellent reliability. There is a need to test diagnostic accuracy of the empirically derived PONS diagnoses in children and adolescents in community child and adolescent mental health and neurodisability services and to assess triaging utility. Future longitudinal studies will report the ability of the PONS to capture change in symptom severity and impairment.
In summary, the PONS scale (parent version) is a HealthTracker™ system-based fast, engaging PCOM with excellent psychometric properties that reports on the frequency and impairment produced by neuropsychiatric symptoms and neurodisability. The PONS-parent version consists of 30 symptom domains, rated on a 7-point scale for frequency and impairment, covers a comprehensive variety of psychopathology - ADHD, ASD, ODD, OCD, Anxiety, Depression, Psychoses and Bipolar Disorder and is especially appropriate for use in those with neurodisabilities. It allows clinicians to profile children and adolescents with emotional and behavioural problems, and could be useful in clinical screening and triaging, and can also be used in clinical trials and epidemiological research. As it is available online with automated scoring and immediate feedback of statistically significant cut-off points, it assists clinicians with diagnostic decision making and optimises use of clinician time. Future research will need to examine PONS as a change measure.
Profile Of Neuropsychiatric Symptoms
Patient Centred Outcome Measures
Patient Reported Outcomes
Suicidality: Treatment Occurring in Paediatrics
Development And Well-Being Assessment
Costello EJ, Egger H, Angold A. 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. Journal of the American Academy of Child and Adolescent Psychiatry. 2005;44(10):972–86.
Waddell C, Offord DR, Shepherd CA, Hua JM, McEwan K. Child psychiatric epidemiology and Canadian public policy-making: the state of the science and the art of the possible. Canadian Journal of Psychiatry. 2002;47(9):825–32.
Maulik P, Mascarenhas M, Mathers C, Dua T, Saxena S. Prevalence of intellectual disability: a meta-analysis of population-based studies. Research in Developmental Disabilities. 2011;32(2):419–36.
Leaf PJ, Alegria M, Cohen P, Goodman SH, Horwitz SM, Hoven CW, et al. Mental health service use in the community and schools: results from the four-community MECA Study. Methods for the Epidemiology of Child and Adolescent Mental Disorders Study. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35(7):889–97.
Meltzer H, Gatward R, Goodman R, Ford T. Mental Health of Children and Adolescents in Great Britain. London: Stationery Office; 2000.
Achenbach T, Rescorla L. The Child Behavior Checklist (CBCL): Manual for the ASEBA Preschool forms and Profiles. Burlington, VT: University of Vermont, Department of Psychiatry; 2000.
Goodman R. The Strengths and Difficulties Questionnaire: a research note. Journal of Child Psychology and Psychiatry. 1997;38(5):581–6.
NIH. 2008; Available from: http://www.nimh.nih.gov/research-priorities/rdoc/index.shtml.
Browne J, Jamieson L, Lewsey J, van der Meulen J, Black N, Cairns J, et al. Patient Reported Outcome Measures (PROMs) in Elective Surgery, 2007: London.
Department of Health. Guidance of the Routine Collection of Patient Reported Outcome Measures (PROMs). London: Department of Health; 2007.
Department of Health. Guidance on the Routine Collection of Patient Reported Outcome Measures (PROMs). For the NHS in England 2009/10. Vol. 2014. London: Department of Health; 2008.
Devlin N, and Sppleby J. Getting the most out of PROMS. Putting health outcomes t the heart of NHS decision-making. 2010: The King’s Fund. Office of Health Economics.
Snyder C, Aaronson N. Use of patient-reported outcomes in clinical practice. The Lancet. 2009;374(9687):369–70.
Valderas J, Alonson J, Guyatt G. Measuring patient-reported outcomes: moving from clinical trials into clinical practice. Medical Journal of Australia. 2008;189(2):93–4.
Vallance-Owen A. Measuring patient care: PROMs promote health gain and patient involvement. British Medical Journal. 2008;336(7640):344.
Haverman L, Engelen V, van Rossum M, Heymans H, Grootenhuis M. Monitoring health-related quality of life in paediatric practice: development of an innovative web-based application. BMC Pediatrics. 2011;11(3):1–7.
Cromby J, Standen P, Brown D. The potentials of virtual environments in the education and training of people with learning disabilities. Journal of Intellectual Disability Research. 1996;40(6):489–501.
Davies D, Stock S, Wehmeyer M. Application of computer simulation to teach ATM access to individuals with intellectual disabilities. Education and Training in Mental Retardation and Developmental Disabilities. 2003;38:451–6.
Harrysson B, Svensk A, Johansson G. How people with developmental disabilities navigate the internet. British Journal of Special Education. 2004;31:138–42.
Wehmeyer M, Smith S, Palmer S. Technology use and people with mental retardation. International Review of Research in Mental Retardation. 2004;31:291–337.
Gringras P, Santosh P, Baird G. Development of an Internet-based real-time system for monitoring pharmacological interventions in children with neurodevelopmental and neuropsychiatric disorders. Child: Care, Health and Development. 2006;32(5):591–600.
Santosh P. STOP study aims to monitor suicidality, 2014: EU Research. p. 36–39.
Courtemanche A, Schroeder S, Sheldon J. Designs and analyses of psychotropic and behavioral interventions for the treatment of problem behavior among people with intellectual and developmental disabilities. American Journal on Intellectual and Developmental Disabilities. 2011;116(4):315–28.
Food and Drug Administration. Guidance for Industry: Patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. 2006 [cited 2014 7 April]; Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1629006/pdf/1477-7525-4-79.pdf.
Bren L. The importance of patient-reported outcomes…It’s all about the patients. In: FDA Consumer. Silver Spring, MD: Food and Drug Administration; 2006.
Nunnally J, Bernstein I. Psychometric Theory. 3rd ed. 1994.
Jaeschke R, Guyatt G. A comparison of seven-point and visual analogue scales: Data from a randomized trial. Controlled Clinical Trials. 1990;11(1):43–51.
Clark A, Watson D. Constructing validity: Basic issues in objective scale development. Psychological Assessment. 1995;7(3):309–19.
Costello A, Osborne J. Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most From Your Analysis. Research and Evaluation: Practical Assessment; 2005. p. 10(7).
Hair J, Tatham R, Anderson R, Black W. Multivariate data analysis. 5th ed. London: Prentice-Hall; 1998.
Reise S, Waller N, Comrey A. Factor Analysis and Scale Revision. Psychological Assessment. 2000;12(3):287–97.
Stevens J. Applied multivariate statistics for the social sciences. 2nd ed. Hillsdale, NJ: Erlbaum; 1992.
Tabachnick B, and Fidell L. Using multivariate statistics. 5th ed. 2007: Pearson Education Inc.
Pappa S, Dazzan P. Spontaneous movement disorders in antipsychotic-naive patients with first-episode psychoses: a systematic review. Psychological Medicine. 2009;39(7):1065–76.
Participants in the research; Dr. Ruksana Ahmed, Consultant Clinical Psychologist; and the Centre for Interventional Paediatric Psychopharmacology (CIPP), Maudsley Hospital, London, UK; and the SME, HealthTracker Ltd for optimizing the scale to be used on their web-based platform.
Dr. Paramala Santosh and Professor Paul Gringras are co-inventors of the HealthTracker™ and Directors of HealthTracker Ltd. Professor Baird, Dr. Fiori and Dr. Sala have no competing interests.
PS designed the PONS and study, helped with data collection and the draft manuscript. PG helped design the PONS, and helped with data collection. GB reviewed the PONS and helped with the draft manuscript. RS and FF helped with the manuscript and statistical analyses. All authors read and approved the final manuscript.
About this article
- Profile Of Neuropsychiatric Symptoms scale
- Patient reported outcomes
- Patient centred outcome measures
- Internet-based scale
- Psychometric properties