Validation of a multifactorial risk factor model used for predicting future caries risk with nevada adolescents
© Ditmyer et al; licensee BioMed Central Ltd. 2011
Received: 30 November 2010
Accepted: 20 May 2011
Published: 20 May 2011
The objective of this study was to measure the validity and reliability of a multifactorial Risk Factor Model developed for use in predicting future caries risk in Nevada adolescents in a public health setting.
This study examined retrospective data from an oral health surveillance initiative that screened over 51,000 students 13-18 years of age, attending public/private schools in Nevada across six academic years (2002/2003-2007/2008). The Risk Factor Model included ten demographic variables: exposure to fluoridation in the municipal water supply, environmental smoke exposure, race, age, locale (metropolitan vs. rural), tobacco use, Body Mass Index, insurance status, sex, and sealant application. Multiple regression was used in a previous study to establish which significantly contributed to caries risk. Follow-up logistic regression ascertained the weight of contribution and odds ratios of the ten variables. Researchers in this study computed sensitivity, specificity, positive predictive value (PVP), negative predictive value (PVN), and prevalence across all six years of screening to assess the validity of the Risk Factor Model.
Subjects' overall mean caries prevalence across all six years was 66%. Average sensitivity across all six years was 79%; average specificity was 81%; average PVP was 89% and average PVN was 67%.
Overall, the Risk Factor Model provided a relatively constant, valid measure of caries that could be used in conjunction with a comprehensive risk assessment in population-based screenings by school nurses/nurse practitioners, health educators, and physicians to guide them in assessing potential future caries risk for use in prevention and referral practices.
Keywordsdental caries DMFT Index Specificity Sensitivity
Although dental caries has declined significantly among school-aged children since the early 1970s, oral disease, including caries, remains a major public health challenge [1–3]. In 2004, the reported prevalence of dental caries was approximately 60% in US children ages 12 to 19, with a reported 20% in untreated tooth decay . Childhood dental caries has been reported to be the most prevalent infectious disease in our nation - 5 times as common as asthma and 7 times as common as hay fever . Sixty-seven percent of 12 to 17 year olds reported caries experience, with more than 7% of all children losing at least one permanent tooth to decay before reaching the age of 17 . Researchers have established associations between poor oral health status and systemic diseases, genetics, behavioral, and environmental factors [5–8]. In an effort to help reduce caries prevalence, adolescents who are at greater risk can be identified through population-based screenings by school nurses/nurse practitioners, health educators, and physicians to guide them in assessing potential future caries risk so referrals can be made to dental professionals. This will also help in targeting intervention strategies, including behavior modifications [i.e., oral hygiene, dietary, fluoridation] . The ability to identify these potential factors such as lifestyle, ethnicity, health status, and social conditions associated with oral health status can help classify, thus distinguish, adolescents who might be at greater caries risk. School nurses/nurse practitioners and health educators are far more likely to encounter adolescents on a regular basis than dental professionals. Therefore, it is essential that they be familiar with the various risk factors associated with dental caries to help make appropriate referral and intervention decisions.
In 2008, a study was conducted to determine the prevalence (untreated and restored lesions and untreated dental caries) and severity (DMFT Indices) among Nevada youth assessed during a statewide, school-based oral health screening initiative, while using the data to develop a theoretical caries risk screening tool for use in population-based screenings that could be validated in future studies. The original study analyzed data from a cohort of students previously screened during 2005/2006 academic school year. Inclusion criteria for participation were parental consent and student assent. The University of Nevada Las Vegas Institutional Review Board approved this initiative to assure student confidentiality.
By validating this instrument, public health practitioners could use the multifactorial Risk Factor Model with other risk assessments with confidence to guide them in prevention and referral practices. Therefore, the objective of this study was to measure the validity and reliability of this multifactorial Risk Factor Model developed for use in predicting future caries risk in Nevada adolescents in a public health setting.
Development of the Multifactorial Risk Factor Model
The original instrument was developed using retrospective data from a school-based, oral health screening initiative consisting of a cohort of 9,202 adolescents between ages 13-19 attending Nevada public/private high schools in the 2005/2006 academic year. Multiple regression analyses were initially used to establish the variables that significantly contributed to caries prevalence and severity (p < 0.05). Follow-up logistic regression ascertained the weight of contribution and odds ratio of the ten significant variables from the multiple regression analyses . These included (in hierarchal order): 1) exposure to fluoridated water in the municipal water supply, 2) exposure to environmental smoke, 3) race, 4) age, 5) locale (metro vs. rural), 6) use of tobacco, 7) Body Mass Index (BMI), 8) dental insurance status, 9) sex, and 10) application of sealants.
Selection of Participants
Researchers examined retrospective data from an oral health surveillance initiative that conducted over 51,000 screenings on children 13 to 18 years of age, attending public/private schools in Nevada across six academic years (2002/2003-2007/2008). The University of Nevada Las Vegas Institutional Review Board approved this study to assure student confidentiality.
The subset of the database was used in the original study  which described the development of the mulitfactorial Risk Factor Model in this study. The oral health screening data were collected by trained, calibrated dentists licensed to practice in the State of Nevada [Nevada Statutes and Regulations, Chapter 631, Dentistry and Dental Hygiene]. Inter-rater and intra-rater reliability between the examiners were computed with intraclass correlation coefficients (ICC) (0.81, p < 0.001 and 0.98, p < 0.001 respectively) .
Defining the Standard Used for Population-based Screening
Comparisons between an existing standard and the proposed Risk Factor Model was established to provide sufficient confidence in the instruments ability to distinguish between those with disease from those without; in this case accuracy of the Risk Factor Model in the prediction of future caries risk in Nevada adolescents in a pubic health setting. Any standard used for this purpose would be defined as a single tool (or combination of) tool(s) in support of that outcome . Because this Risk Factor Model was designed to be used in population-based screenings, the gold standard selected for establishing caries prevalence and disease status was the DMFT (decayed, missing, and filled teeth) Index that was developed in 1938 by Klein et al. . Historically, surveys using the Decayed, Missing and Filled Tooth (DMFT) Index have provided data that was translated into future predictions, such as the potential number of teeth to be restored and extracted based on the strength of impact imposed by risk factors identified [14–18]. A review of the literature found that past caries experience was confirmed as the most significant predictor of future caries development, even beyond that of bacterial and socioeconomic factors . The DMFT Index is the criterion still used in large-scale national population-based studies, such as National Health and Nutrition Examination Survey [NHANES] to define caries prevalence and severity. Individuals with a DMFT Index ≥1 are classified as 'with disease' while those with a DMFT Index = 0 are classified 'without disease' [3, 13]
An important challenge in screening practices in the public health arena is the ability to correctly identify individuals who do and do not have a disease [19–21]. The screening should be defined as a means for early detection and referral for potential treatment of a disease that is available to a population. This process includes the screening and the follow-up evaluation for those who are considered high risk [19–21]. Characteristics of a successful screening include low cost, minimal risk, convenience, validity, and reliability. Therefore the screening must have a high degree of reliability and validity. This provides a basis for targeting at-risk populations for primary prevention.
Consensus recommendations published by National Institutes of Health stated that individuals with moderate to high risk for dental disease should be identified as early as possible so aggressive strategies could be adopted . Risk Factor scores were calculated for all subjects using the Multifactorial Risk Factor Model parameters . In keeping with NIH criteria for management of caries, operational definitions were created. Individuals with moderate Risk Factor scores (2.7-4.4) and high Risk Factor scores (≥4.5) were collapsed into one category and identified as testing 'positive'. Those with low Risk Factor scores (≤2.6) were considered as testing 'negative'.
The key parameters used in defining the utility of this Risk Factor Model included sensitivity, specificity, predictive value positive (PVP), predictive value negative (PVN), and disease prevalence [19–22]. When attempting to assess biologic variations within human populations, the tools used should distinguish between those with normal and those with abnormal results and thus allow for a better understanding of how different characteristics are distributed in the population being studied . Therefore a screening tool (Risk Factor Model) is expected to adequately distinguish between those who have the disease [or characteristic] (sensitivity) from those who do not have the disease (or characteristic) (specificity) [19–22].
Screenings were evaluated for feasibility [19–22], with measures that included: predictive value positive (PVP), and predictive value negative (PVN). PVP is defined as the proportion of individuals who tested positive and actually had the disease (or characteristic) at the time screening was conducted. A high PVP implies that the screening program is effective because it detects a large proportion of actual cases among individuals with positive results. PVN is defined as the proportion of individuals who tested negative and were without the disease (or characteristic). It should be noted, that for any population-based screenings, the PVP drops as the prevalence of the disease decreases. Conversely, the PVN rises as the prevalence of the disease decreases. Low prevalence rates generally infer that persons being tested do not have the disease. This dependency on accurate prevalence rates required the use of a representative sample of the population studied and was necessary for calculating predictive values. Direct measures by the licensed dentists in this initiative allowed for sufficient confidence in identification of cases of dental caries. Subsequently, direct measures of prevalence rates (DMFT indices) were computed using actual screening data, thus considered valid for the purposes of identifying dental caries prevalence [3, 10].
Classification of Data for Reliability and Validity Analyses (Two-by-Two Computation Table)
DMFT Score ≥ 1.0
DMFT Score = 0
High Caries Risk
[Moderate + High Risk Score]
True Positive [TP]
False Positive [FP]
Total test positives
a + b
Low Caries Risk
[Low Risk Score]
False Negative [FN]
True Negative [TN]
Total test negatives
c + d
Total with disease
a + c
Total without disease
b + d
a + b + c + d
Predictive Value Positive
Predictive Value Negative
Disease classification two-by-two table results of subjects' classifications for each academic year
With Disease DMFT Score > 1.0
Without Disease DMFT Score = 0
High Caries Risk
Low Caries Risk
High Caries Risk
Low Caries Risk
High Caries Risk
Low Caries Risk
High Caries Risk
Low Caries Risk
High Caries Risk
Low Caries Risk
High Caries Risk
Low Caries Risk
Summary Table of Computations by Academic Year for Specificity, Sensitivity, PVP, PVN, and Prevalence
Predictive Value %
Sensitivity % [95% CI]
Specificity % [95% CI]
Positive [95% CI]
Negative [95% CI]
Prevalence % [95% CI]
The purpose of this study was to determine whether the Risk Factor Model previously developed was a valid instrument that could be used by non-dental professionals in public health settings for predicting future caries risk among Nevada adolescents and that could lead to appropriate prevention and referral practices. In practical terms, the predictive capacity of the Risk Factor Model was not very different across the six years. Differences in yearly variability can be attributed to two factors: 1) differences in the prevalence rates from year-to-year, and 2) high reported prevalence rates. Subjects' overall mean caries prevalence across all six academic years was 66% with a range reported from 62% in year 3 (2004/2005) to 71% in year 2 (2003/2004). These differences could have resulted in the variability found in sensitivity, specificity, PVP and PVN from year-to-year . Differences in these predictive values could also have been attributed to the sample's high caries prevalence rates. This study confirmed that dental caries remained a common chronic disease among Nevada youth during the years studied, presenting with a higher prevalence rate than the national average (66% vs. 59%) for adolescents and higher than the Healthy People 2010 goal for Nevada of 51% [3, 25]. This study confirmed average prevalence rates (66%) were representative of the Nevada adolescent population reporting 61% prevalence in 2006 .
Although direct measures were obtained by licensed dental professionals, self-report information of demographic variables in the initial oral health screening warrants some caution in interpreting data. Data collection and screening protocols were documented extensively with quality control guidelines in place. Due to student confidentiality issues, students were not tracked over time; however collection of cross-sectional data of Nevada youth from all demographics across all six years helped strengthen interpretations and provided a strong representation of the Nevada adolescent population.
The cut-off values used in this study were adopted from a previous study conducted in the same population . It should be noted that cut-off values for the various categories (low, moderate, and high risk) could vary in different populations. For instance in some populations 90% of subjects may be low risk if a DMFT <2.6 is used while in other populations only 10% may fall in that category. Therefore, caution should be used when generalizing to different populations.
Conclusion and Clinical Relevance of Findings
These results indicated that this Risk Factor Model could be used to predict future caries risk with sufficient confidence in conjunction with other risk assessments by non-dental health practitioners, such as school nurses/nurse practitioners, health educators, and physicians in population-based settings. Interpretation of epidemiological data by investigators is generally considered subjective. A review of the literature showed a wide range of acceptable levels regarding sensitivity, specificity, predictive value positive, and predictive value negative [12, 26–29]. Ranges from 70% to 100% have been considered good to very good or valid levels. Defining validity of an instrument has always required that many factors be considered. Standards to assess the validity of tests used by dental professionals for diagnostic purposes must be set higher than standards set forth in population-based screenings used by non-dental health practitioners to guide them in assessing potential future caries risk for use in prevention and referral practices. Thus, such a Risk Factor Model should be looked at from the 'practicality' aspect for its intended purpose.
Previous studies of school-based dental screening have found it to stimulate follow-up for dental services of those socially deprived . What must be considered is the effectiveness of population-based screenings of the number identified as high risk who subsequently go on to receive appropriate treatment. There is some evidence to suggest that children who are identified for follow-up services after school-based dental screenings leads to improved attendance with dental professional .
Because non-dental healthcare professionals (e.g., such as school nurses/nurse practitioners, health educators, and physicians, etc.) have frequent contact with adolescents and their parents/guardians, during health screenings at school or in the community, a valid Risk Factor Model is an excellent tool that can be used in conjunction with other risk assessments to help guide early detection, prevention, and referral practices. Results indicated that prediction success was relatively high, and the Risk Factor Model was found to have satisfactory predictive power. Thus, this study provided sufficient confidence in the Risk Factor Model as a population-based screening measure and indicates that the Risk Factor Model may be useful in public health practice as a reliable scale for predicting future caries risk for early detection and referral practices by non-dental health practitioners. Future prospective studies designed to examine the practical use of the Risk Factor Model by Nevada non-dental healthcare professionals could further support these findings.
Source of Funding: Crackdown on Cancer Oral Health Surveillance Screening Program receives funding through the Trust Fund for Public Health and The Fund for Healthy Nevada, both beneficiaries of the Master Tobacco Settlement.
Thank you to: Dr. Christina Demopoulos, Director of Crackdown on Cancer Oral Health Surveillance Screening Program
- Peterson PE, Bourgeois D, Ogawa H, Estupinan-Day S, Ndiaye C: The global burden of oral diseases and risks to oral health. Bull of the World Health Organ. 2005, 83 (9): 661-669.
- National Institute of Dental and Craniofacial Research: A National Call to Action to Promote Oral Health. The Office of the Surgeon General, [cited 2006 July 12]. Available from, [http://www.cdc.gov/OralHealth/publications/factsheets/call_to_action.htm]
- U.S. Department of Health and Human Services: Trends in Oral Health Status: United States, 1988-1944 and 1999 to 2004. Preliminary Report. Vital and Health Statistics. 2007, DHHS Publication No. [PHS] 2007-1698, Hyattsville, MD, 11 (248):
- Council of State Governments' (CSG) Healthy States Initiative: Promoting Improved Oral Health. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, under Cooperative Agreement U38/CCU424348. 2008, Available from, [http://www.healthystates.csg.org/NR/rdonlyres/7977906A-8420-4770-A5C7-5173952742CD/0/OralHealthLPB_screen.pdf]
- U.S. Department of Health and Human Services: Oral health in America: A Report of the Surgeon General. 2000, DHHS, Rockville, MD
- Reisine ST, Psoter W: Socioeconomic status and selected behavioral determinants as risk factors for dental caries. J Dent Educ. 2001, 65: 1009-16.PubMed
- Reid BC, Hyman JJ, Macek MD: Race/ethnicity and untreated dental caries: the impact of material and behavioral factors. Community Dent Oral Epidemiol. 2004, 32: 329-36. 10.1111/j.1600-0528.2004.00165.x.View ArticlePubMed
- Kiess W, Galler A, Reich A, Müller G, Kapellen T, Deutscher JK: Clinical aspects of obesity in childhood and adolescence. Obesity Reviews. 2001, 2 (1): 29-10.1046/j.1467-789x.2001.00017.x.View ArticlePubMed
- National Institutes of Health: Diagnosis Management of Dental Caries throughout Life. NIH Consensus Statement. 2001, 181: 1-30.
- Ditmyer MM, Mobley C, Draper Q, Demopoulos C, Smith ES: Development of a Theoretical Screening Tool to Assess Caries Risk in Nevada Youth. J Public Health Dent. 2008, 201-208.
- Shrout PE, Fleiss JL: Intraclass correlationas: uses in assessing rater reliability. Psychol Bull. 1979, 86: 420-8.View ArticlePubMed
- Parikh R, Mathai A, Parikh S, Sekhar GC, Thomas R: Understanding and using sensitivity, specificity, and predictive values. Indian J Ophthalmol. 2008, 56: 45-50. 10.4103/0301-4738.37595.PubMed CentralView ArticlePubMed
- Klein H, Palmer CE, Knutson JW: Studies on dental caries. Dental status and dental needs of elementary school children. Pub Health Rep. 1938, 53: 751-765.View Article
- Bader JD, Perrin NA, Maupome G, Rush WA, Rindal BE: Exploring the contribution of components of caries risk assessment guidelines. Community Dent Oral Epidemiol. 2008, 36: 357-62. 10.1111/j.1600-0528.2007.00399.x.View ArticlePubMed
- Spencer AJ: Preliminary investigation of the relationship between population DMFT scores and mix of restorative and exodontic needs. Community Dent Oral Epidemiol. 1981, 9: 215-18. 10.1111/j.1600-0528.1981.tb00332.x.View ArticlePubMed
- Powell LV: Caries prediction: A review of the literature. Community Dent Oral Epidemiol. 1998, 26: 361-71. 10.1111/j.1600-0528.1998.tb01974.x.View ArticlePubMed
- David J, Astrom AN, Want NJ: Prevalence and correlates of self-reported state of teeth among schoolchildren in Kerala, India. BMC Oral Health. 2006, 6-
- Motohashi M, Yamada H, Fugio G, Hideyuki K, Toshiko I, Setsuko S, Atsuto S, Masao M: Employing dmft score as a risk predictor for caries development in the permanent teeth in Japanese primary school girls. Jour Oral Sc. 2006, 48 (4): 233-7. 10.2334/josnusd.48.233.View Article
- Gordis L: Epidemiology. 2008, Philadelphia, PN: Saunders Publishing Company, 4
- Aschengrau A, Seage GR: Essentials of Epidemiology in Public Health. 2008, Boston, MA: Jones and Bartlett Publishers
- Szklo M, Nieto FJ: Epidemiology: Beyond the Basics. 2007, Boston, MA: Jones and Bartlett Publishers
- Loong TW: Understanding sensitivity and specificity with the right side of the brain. BMJ. 2003, 327: 716-9. 10.1136/bmj.327.7417.716.PubMed CentralView ArticlePubMed
- Hinkle DE, Wiersma W, Jurs SG: Applied Statistics for the Behavioral Sciences. 2003, Boston, MA: Houghton Mifflin Company
- Oral Health Program, Nevada State Health Division: The Burden of Oral Disease in Nevada. 2006, Available from, [http://health.nv.gov/PDFs/OH/Burdenoforaldisease2006.pdf]
- Pinelli C, de Castro L, Loffredo M: Reproducibility and validity of self-perceived oral health conditions. Clin Orl Invest. 2007, 11: 431-437. 10.1007/s00784-007-0133-0.View Article
- Beltran ED, Malvitz DM, Eklund SA: Validity of two methods for assessing oral health status of populations. J Pub Health Den. 1997, 57 (4): 206-14. 10.1111/j.1752-7325.1997.tb02977.x.View Article
- Pietrla T, Pietila I, Vaataja P: Early screening for orthodontic treatment. Differences in assessments made by a consultant orthodontist and three public health dentists. Community Dent Oral Epidemiol. 1992, 20: 208-13. 10.1111/j.1600-0528.1992.tb01718.x.View Article
- Pitiphat W, Garcia RI, Douglass CW, Joshipura KJ: Validation of self-reported oral health measures. J Public Health Dent. 2002, 62: 122-8. 10.1111/j.1752-7325.2002.tb03432.x.View ArticlePubMed
- Nilsson IM, List T, Drangsholt M: The reliability and validity of self-reported temporomandibular disorder pain in adolescents. J Orofac Pain. 2006, 20: 138-44.PubMed
- Milsom K, Threlfall A, Blinkhorn A, Kearney-Mitchell P, Buchanan K, Tickle M: The effectiveness of school dental screening: dental attendance and treatment of those screened positive. Br Dent J. 2006, 687-690.
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6831/11/18/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.