Skip to main content
Download PDF

Pediatric dentistry systematic reviews using the GRADE approach: methodological study

BMC Oral Health volume 24, Article number: 787 (2024) Cite this article

Abstract

Background

To assess the reporting of the certainty of the evidence using the GRADE approach in systematic reviews of interventions in pediatric dentistry.

Methods

The inclusion criteria were systematic reviews of randomized clinical trials (RCTs) and non-randomized studies of interventions (NRSIs) in pediatric dentistry that reported the certainty of the evidence through the GRADE approach. Paired independent reviewers screened the studies, extracted data, and appraised the methodological quality using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR 2) tool. The certainty of the evidence was extracted for each outcome. A descriptive analysis was conducted.

Results

Around 28% of pediatric dentistry reviews of interventions used the GRADE approach (n = 24). Twenty reviews reported 112 evidence outcomes from RCTs and 13 from NRSIs using GRADE evidence profile tables. The methodological quality was high (16.7%), moderate (12.5%), low (37.5%), and critically low (33.3%), fulfilling the majority of the AMSTAR 2 criteria. The certainty of the evidence for outcomes generated from RCTs and NRSIs was very low (40.2% and 84.6%), low (33.1% and 7.7%), moderate (17.8% and 7.7%), and high (9.8% and 0.0%). The main reasons to downgrade the certainty were due to (for RCTs and NRSIs, respectively): risk of bias (68.8% and 84.6%), imprecision (67.8% and 100.0%), inconsistency (18.8% and 23.1%), indirectness (17.8% and 0.0%), and publication bias (7.1% and 0.0%).

Conclusion

The proportion of systematic reviews assessing the certainty of the evidence using the GRADE approach was considered small, considering the total initial number of published pediatric dentistry reviews of intervention. The certainty of the evidence was mainly very low and low, and the main problems for downgrading the certainty of evidence were due to risk of bias and imprecision.

Registration

PROSPERO database #CRD42022365443.

Peer Review reports

Background

Evidence-based dentistry has contributed substantially to improve oral health quality in general and the pediatric population more specifically. A systematic survey published in 2014 presented high-quality Cochrane systematic reviews (SRs) in pediatric dentistry [1]. However, that review did not evaluate the certainty of evidence, which has only more recently been analyzed methodologically in medicine and dentistry [2, 3]. In the medical field, only 43.6% of Cochrane reviews assessed the GRADE approach (Grading of Recommendations, Assessment, Development, and Evaluation) using a summary of findings (SoF) Table [3]. In dentistry, Cochrane reviews were 12 times more likely to assess the certainty of evidence in comparison to non-Cochrane reviews [2]. Recent methodological surveys in dentistry found mainly low and critically low-quality SRs [4,5,6]. Among them, one survey found that 54.3% of SRs in orthodontics assessed the certainty of the evidence through the GRADE approach, and 34.8% of those studies correctly followed all GRADE criteria [4]. In periodontology, 25.2% of SRs included the GRADE approach, and 37.5% of those studies followed all the GRADE criteria [5].

A systematic and explicit approach of study evidence can help prevent errors, facilitate critical assessment, and improve information dissemination. The GRADE approach enables more consistent analysis of the evidence, and subsequent communication can support better-informed choices in health care [7].

By assessing the methodological quality of SRs and the certainty of evidence, it is possible to identify gaps for improvement in these studies. A methodological survey published in 2015 found that most of the evidence in pediatric dentistry was low or very low [8]. Only 15 SRs on fluorides and caries prevention reported high to moderate evidence of 81 SRs evaluated. By searching for gaps through methodological surveys, we can identify which areas of pediatric dentistry need more research and which have provided high-quality evidence, resulting in an efficient use of time, money, and research effort. Therefore, this methodological survey aims to evaluate the certainty of evidence using the GRADE approach in pediatric dentistry.

Methods

Eligibility criteria

This methodological study included SRs and meta-analyses of interventions in pediatric dentistry published from January 1st, 2020 to March 3rd, 2022. All studies published during this period were considered for inclusion, characterizing a convenience sample. The study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) [9].

To be considered a SR, the following criteria had to be fulfilled [6, 10]: clear clinical question with the PICO acronym (patient, intervention, comparison, outcome), clear systematic search, reproducible methodology, assessment of risk of bias, systematic presentation and synthesis of results. For inclusion, SRs should have included randomized controlled trials (RCTs) or non-randomized studies of interventions (NRSIs) that reported the certainty of the evidence using the GRADE approach. Included original studies had to be related to pediatric treatments in babies, children, or adolescents. There was no limitation regarding the language of publication.

Exclusion criteria: reviews including original studies with individuals older than 18 years, treatments or outcomes in other fields (e.g., orthodontic, endodontics in permanent teeth, surgery); other types of SRs/meta-analyses (observational studies of exposures related to the outcome (PECO), of in vitro/animal studies, and others), guidelines, scoping reviews, overviews (reviews of reviews), narrative reviews, letters/editorials of SRs, letters to the editor, cases/case series, other original studies (such as in vitro, animal studies, observational and clinical studies), SRs not reporting the GRADE approach and those using different approaches for assessment of the level or quality of evidence, review protocols, network meta-analyses.

Information source and search strategy

Five electronic databases were searched: MedLine, Embase (both through Ovid), Cochrane Database of Systematic Reviews, Scopus and Web of Science. Supplementary Table S1 shows detailed search strategies for each database. Subsequently, the studies were organized in EndNote software, version X9.3.1 (Philadelphia, PA: Clarivate Analytics) for the removal of duplicates.

Study selection

Independent paired reviewers screened titles and abstracts (SQN/JRC and ACSO/LPS) based on the predefined eligibility criteria using the Rayyan QCRI website. The same reviewers independently screened the full texts. Before each phase, the senior reviewer (CCMP) trained the reviewers with 10% of the retrieved studies. In all phases, consensus was reached through discussion. During the screening processes, disagreements were resolved through consensus by the pair of reviewers and if not achieved, by consulting the senior reviewer (CCMP).

Data collection and data items

Independent paired reviewers (SQN/LPS, SQN/AGP, and JRC/ACSO) extracted data using a spreadsheet created on Excel software. The expert reviewer (SQN) trained the reviewers before extraction, using 5% of the included studies to guarantee uniformity and consistency during data extraction. During the extraction process, the pair of reviewers discussed and solved disagreements by consensus. When a consensus was not reached, the expert reviewer gave the final vote (SQN). All reviewers involved in the data collection had previous experience in data extraction and authored at least one SR.

The extracted data included the following information: authors, the geographical location of the authors, the journal’s name, impact factor from the Journal of Citation Reports, the research topic, the language of the publication, the included studies’ design, the number of primary studies, the number of RCTs and NRSIs, the total number of patients, the number of studies included in the meta-analysis, the risk of bias tool and the presence of a meta-analysis. The research topic was grouped according to Mejare et al. [8]: treatment and prevention of dental caries (fluorides, restorations and sealants), plaque control and oral hygiene education (toothbrushing, antimicrobials, mouthwashes and education programs), anesthesia (solutions, techniques), endodontics (rotatory and manual techniques), treatment of oral lesions (ankyloglossia and mucositis), behavior management/anxiety (techniques and technologies), and dental eruption.

One reviewer (RAB), experienced in assessing the certainty of the evidence through the GRADE approach, extracted the certainty of the evidence. For data extraction, we considered only reviews that presented the certainty of the evidence through a GRADE evidence profile table or summary of findings (SoF) table. The following variables were extracted for each outcome: the intervention and comparison, type of outcome, the effect estimate and confidence interval (95%CI), the direction of the effect (favoring the intervention or control), the final rating for each GRADE domain (no problem, serious, or very serious [for risk of bias, inconsistency, indirectness, imprecision, publication bias]), upgrading the certainty for dose-response, large effect and residual confounders (for NRSI). The final certainty was extracted per outcome (high, moderate, low, and very low) [11]. The same reviewer cross-checked data extraction one month later to minimize extraction errors.

The corresponding authors of the included studies were contacted once through email to clarify details when required.

Methodological quality

Independent and trained paired reviewers (SQN/LPS, AGP/LPS, and JRC/ACSO) assessed the methodological quality of the included reviews using the AMSTAR 2 tool [12]. The expert reviewer trained the reviewers and discussed any disagreements until they reached a consensus (likewise regarding the description for data extraction). At least one of the paired reviewers had previous experience with AMSTAR 2 and authored at least one methodological survey (SQN, AGP, JRC).

AMSTAR 2 has 16 items and four criteria, each judged as “yes” (no critical weakness), “partial yes” (non-critical weakness), “no” (critical flaw), and “no meta-analysis conducted.” We followed the seven critical items according to AMSTAR 2 to rate the final quality (2, 4, 7, 9, 11, 13 and 15) [12]. The final quality of each SR was “high” when there was no or one “non-critical weakness” on one of the critical item; “moderate” when there was more than one “non-critical weakness;” “low” when there was one “critical” flaw with or without a “non-critical weaknesses,” or “critically low” when there was more than one “critical flaw” with or without “non-critical weaknesses.” For each AMSTAR 2 item, we grouped “yes” and “partial yes” into “no critical weaknesses” and “no” as “critical flaw.”

Summary measurements and synthesis of results

The data were imported into IBM SPSS Statistics for Windows, Version 25 (Armonk, NY: IBM Corp.). Categorical variables were descriptively presented as absolute and relative frequencies, while numeric variables (e.g., total number of patients and included studies) were summarized using means and standard deviations. The Person chi-square test was used to test the association between methodological quality and the certainty of the evidence.

Results

Study selection

The electronic and manual searches retrieved 7,198 records. A total of 3,531 titles and abstracts of studies were selected after removing duplicates. One hundred thirty-four full texts were included for analysis. Finally, 24 studies that reported the certainty of the evidence through the GRADE approach were included (Fig. 1. PRISMA flowchart, Supplementary Table S2). Supplementary Table S3 shows the list of excluded studies and reasons for exclusion.

Fig. 1
figure 1

PRISMA flowchart showing the screening process

Full size image

Study characteristics

Table 1 shows the reviews’ characteristics. All reviews were published in English (100%), and one-third had authors from different countries collaborating with the research team (33.3%). Only four of 24 were Cochrane reviews.

Table 1 Frequency distribution of reviews’ characteristics
Full size table

The SRs included a mean of 12.4 original studies (standard deviation – SD: 7.6). All SRs included RCTs and four SRs included NRSIs and RCTs. The mean number of RCTs included was 10.5 (SD: 4.9). Nineteen reviews had meta-analysis (79.2%).

All the reviews used the original Cochrane Risk of Bias tool (58.3%) or the RoB 2 (41.6%) for the risk of bias of RCTs. Two reviews that included NRSIs used the ROBINS-I, one used MINORS and one used RoB 2.

Methodological quality of included reviews

Table 2 shows the distribution of the methodological quality per research topic. The methodological quality assessed by AMSTAR 2 was high (n = 4, 16.7%), moderate (n = 3; 12.5%), low (n = 9, 37.5%), and critically low (n = 8, 33.3%). The most explored topic was the treatment and prevention of dental caries [11 reviews: 36.4% being critically low (n = 4), 54.5% low (n = 6), and one moderate quality], followed by plaque control and oral hygiene education (four reviews: one being critically low, one low, one moderate and one with high methodological quality).

Table 2 Distribution of methodological quality (AMSTAR 2) according to the research topic
Full size table

Table 3 shows the quality assessed by AMSTAR 2. The majority of the reviews fulfilled most of the “no critical weaknesses” of AMSTAR 2. More than 90% of the reviews reported the appropriate inclusion of PICO components in the research question and inclusion criteria (item 1, 95.8%); explained the selection of study designs for inclusion in the review (item 3, 91.6%); reported a comprehensive literature search (item 4, 95.8%); reported duplicate and independent screening of studies (item 5, 91.6%); included an adequate description of studies’ characteristics (item 8, 100.0%); used a satisfactory tool for assessing the risk of bias (item 9, 100.0%); considered the risk of bias when interpreting the results (item 13, 91.6%); discussed the observed heterogeneity in the review results (item 14, 91.6%); and reported conflict or interests and funding of the review (item 16, 91.6%). Conversely, 79.1% of the reviews did not report the funding source for the studies included in the SR (item 10). The following items were responsible for the low or critically low quality of the reviews: item 2) 29.1% of reviews did not report the protocol before conducting the review nor deviations from the protocol; item 7) 45.8% of the reviews did not provide the list of excluded studies nor reasons for exclusion; item 15) 25.0% of reviews did not investigate publication bias nor explored its impact upon the results of the review.

Table 3 Frequency distribution of AMSTAR 2 items among reviews
Full size table

Reporting the certainty of the evidence through the GRADE approach

Among 24 SRs that reported the GRADE approach, four SRs were excluded from the analysis, as two reviews did not present a GRADE evidence profile or SoF table with the certainty of the evidence [14, 15]; one review assessed the certainty per study (not per outcome) [16]; and one review combined RCTs and NRSIs to generate the evidence [17]. Consequently, we analyzed the certainty of the evidence for 20 reviews reporting 125 outcomes (112 outcomes generated from RCTs and 13 from NRSIs). Mainly, the certainty was low (33.1% and 7.7%) and very low (40.2% and 84.6%) for outcomes from RCTs and NRSIs, respectively (Tables 4 and 5).

Table 4 Distribution of the certainty of the evidence according to the research topic
Full size table
Table 5 Distribution of the certainty of the evidence (final rating of GRADE approach) according to the methodological quality judged by AMSTAR 2
Full size table

There were only 10 high-certainty outcomes (8.9% - RCT evidence) (Table 4). Among them, six favored the intervention: virtual reality eyeglasses were more effective than other distraction techniques for anxiety during restoration, behavior during caries removal, pain during restoration (behavior management/anxiety topic) [18], quality of obturation favoring the rotatory instrument versus manual (endodontics topic) [21], and less pain for articaine versus lidocaine (anesthesia topic) [22]. Another four high-certainty outcomes favored the intervention or were similar to the comparison group, in other words, the 95%CI crossed the null effect line though the effect estimated favored the intervention (behavior management/ anxiety topic) [18].

Yet, 20 outcomes were considered moderate (17.9%), and 10 showed more efficacy of the intervention than the comparison group: pain, instrumentation time [19, 21], and quality of obturation favoring rotatory instrument (endodontics topic) [21]; microorganisms count favoring synthetic antimicrobial [23]; dental plaque favoring modified toothbrushes (plaque control and oral hygiene education) [13]; and control of radio/chemotherapy-induced oral mucositis favoring honey (treatment of oral lesions) [24]. The remaining outcomes had CIs crossing the null effect line, indicating superior or similar efficacy favoring the intervention (Table 4).

Among NRSIs, there was only one moderate-certainty outcome (gingival health for rehabilitation with zirconia crown compared to stainless steel crown [treatment and prevention of dental caries]) [25]. Supplementary Table S4 lists the outcomes with very low, low, moderate and high certainty, the effect estimates with respective 95%CI, and the direction of the effect.

Table 5 shows the association between the methodological quality - as measured by AMSTAR 2 - and the certainty of the evidence – as measured by the GRADE approach. Based on Pearson chi-square tests, there was a statistically significant association between methodological quality and the certainty of the evidence for RCTs (p < 0.001) and NRSIs (p < 0.002).

Risk of bias was the main reason for downgrading the certainty of the evidence (serious problem: 55.4% among RCTs and 84.6% among NRSIs), followed by imprecision (53.5% among RCTs and 100% of NRSIs) (Table 6). There were serious problems of inconsistency (18.8% and 23.1% of the outcomes generated from RCTs and NRSIs) and serious problems of indirectness in 17.8% of outcomes from RCTs.

Table 6 Frequency and absolute distribution of overall and per domain GRADE rating evaluated per outcome (N: number of outcomes)
Full size table

The certainty of the evidence was not upgraded for any outcome (NRSIs evidence).

Discussion

This survey found the methodological quality of systematic reviews in pediatric dentistry was mostly low or very low. Another survey evaluating pediatric dentistry reviews found similar results. That study, published in 2015, found a 53% high risk of bias in SRs [8]. However, the low methodological quality is not an exclusive characteristic of pediatric dentistry SRs. Other dentistry, orthodontics, and periodontology surveys also considered the SRs in these areas mainly low and critically low quality, using AMSTAR 2 [4,5,6]. Similar findings are also common in medicine, such as post-interventions after breast cancer, treatment of major depression, and living SRs of COVID-19 – all presenting mainly low and critically low quality [26,27,28]. Even the updated SR does not seem to improve compared to the original one, as found by a study comparing the quality of the original and updated non-Cochrane reviews. That survey found that all original SRs were low and critically low quality, and there was no significant difference in the methodological quality comparing the original SRs with the updated version [29]. Other surveys found that Cochrane reviews had higher quality than non-Cochrane reviews [1, 2]. However, these surveys date from seven years ago. Another survey included SRs in dentistry, published from 1980 to 2014, and concluded that the methodological quality increased over the years, especially after the 2000’s [30].

Four AMSTAR 2 items were the most common sources of methodological problems (items 2, 7, 10 and 15). Three of these are considered “critical” items that might have influenced the low and critically low quality (2, 7 and 15). Around one-third of reviews did not report the protocol before conducting the review (item 2), indicating a general neglect of the research protocol. Documenting the protocol before conducting the review is important to minimize potential bias during the review process. For example, prior knowledge of the evidence may influence the definition of a systematic question, the choice for eligibility criteria, or the prespecification of intervention comparisons and outcome analysis [31]. We speculate that deviation from the protocol may be higher than reported. Another critical concern is simple to cover but neglected, as indicated in item 7: 45.8% of reviews did not provide the list of excluded studies with reasons. Authors could publish lists of excluded studies in the accompanying supplementary material. Lastly, about one-quarter of reviews failed to analyze the publication bias and its impact on the results (item 15). Moreover, item 10 could help address item 15. Item 10 relates to reporting the funding sources of the included studies in the SR. The funding source of the included studies in the SR can help resolve conflicts of interest and publication bias. 79% of SRs did not report the funding sources of included studies within the SR. A previous study found that many meta-analyses in pediatric dentistry did not adequately assess the possibility and impact of reporting biases, especially publication bias, which might influence results and conclusions [32]. Funnel plots, Begg’s and Egger’s tests, and identifying for-profit organizations associated with studies can help to reveal publication bias. The GRADE approach suggests downgrading the certainty of the evidence in these situations [33]. The investigation of industry funding in medicine meta-analysis results is better recognized [34, 35] but still lacking in dentistry [32, 36]. Coincidence or not, we found that only 7.1% of outcomes that originated from RCTs had suspected publication bias (or someone could speculate that the reviews failed to analyze and identify publication bias properly). In addition, it is important to highlight that the funding of included studies (item 10) differs from funding and conflict of interest of the systematic review (item 16). The review adequately addressed this last item (item 16 was covered by 91.6% of reviews). To summarize, for-profit funding should be investigated in SRs of interventions.

The certainty of the evidence was generally low and very low, and the main problems for downgrading the certainty of the evidence were due to risk of bias and imprecision. About 19% of SRs in dentistry published in 2017 reported the GRADE approach [37]. In orthodontics and periodontology, the proportion of SRs of interventions that included the GRADE approach was 54.3% and 25.2%, respectively [4, 5]. A 2015 systematic map of pediatric dentistry reviews reported that many studies did not use the GRADE approach, in agreement with our study [8]. Nine years separated our survey from the previous one, and still, the proportion of pediatric dentistry reviews using the GRADE approach is low. We excluded 63 SRs because they did not include the GRADE approach. Therefore, we could estimate around 28% of SRs reporting the GRADE approach in the present study [(24 of (63 + 24 = 87), Fig. 1]. On a positive note, of those SRs that incorporated the GRADE approach, 83.3% followed the GRADE criteria through a GRADE evidence profile or SoF Table (20 of 24).

Similar to another survey in dentistry, the main problems for downgrading the certainty were risk of bias and imprecision [2]. The SRs included the proper risk of bias tools according to the study design, as the RoB 2 and ROBINS-I were predominantly used for RCTs and NRSIs, respectively. Also, when correctly assessed, the risk of bias may facilitate the assessment of the certainty of the evidence. Similarly, imprecision might be easily detected, especially for evidence generated from meta-analysis where the 95%CI crosses the null effect line. Notably, about 50% of the moderate evidence was downgraded due to imprecision because the 95%CI crossed the null effect line. In this case, it means moderate evidence that the intervention was similar to or superior to the control. However, caution should be taken, as the authors did not evaluate the evidence based on the magnitude of the effect or based on the minimal important difference, as recently recommended by the GRADE approach [38].

Ten outcomes had high certainty of evidence. It means that the true effect lies very close to the observed effect, and future research can hardly change the evidence [11]. However, 4 of 10 high-certainty outcomes had the 95%CIs crossing the null effect line. This imprecision means that the intervention and the control have similar effects. Reliable evidence for decision-making should consider the following criteria: (1) the magnitude of the effect or if the desirable health effect (or benefit) is substantial; (2) if the undesirable health effect (or harms) is substantial; (3) the variability or uncertainty about the values (confidence interval addressing imprecision); (4) the certainty of the evidence [39]; and we would add (5) whether the evidence is generated from high-quality SR - or at least moderate-quality SR. We found 17 high-to-moderate certainty of evidence outcomes from high-to-moderate-quality reviews (15%, Table 5). The most reliable evidence for decision-making was the moderate-quality SR that reported high certainty of the evidence for virtual reality glasses for behavior, anxiety, and pain management (3 outcomes) [18]. Similarly, there is moderate certainty for the efficacy of rotatory instruments versus manual instruments at 6 and 48h of follow-up and instrumentation time (3 outcomes) [19] and moderate certainty of the efficacy of the theory-guided intervention compared to conventional education session for oral hygiene education (1 outcome) [20] – these outcomes were from high-quality reviews. The other high and moderate certainty outcomes had 95%CI crossing the null effect line or were from SRs presenting low or critically low methodological quality. Likewise, all NRSIs presented low or critically low quality, though one outcome contained moderate certainty of evidence. Our results are similar to a previous methodological survey that found mainly low or very low evidence in pediatric dentistry SRs. In that survey, 15 SRs reported high to moderate certainty of evidence for fluoride technologies (or other technologies) for caries prevention [8]. Contrary to the previous study, we did not find high or moderate evidence for the treatment and prevention of dental caries.

In summary, low and very low evidence were prominent in this survey. Regardless of whether the outcome had problems of risk of bias, inconsistency, indirectness, imprecision or publication bias, it is highly probable that future research will change the evidence published by these SRs [11]. Therefore, future clinical trials and reviews in these areas are welcome.

Strengths and limitations

The study is strong in reporting the certainty per outcome and per GRADE domain. Furthermore, we analyzed the methodological quality and the certainty of the evidence per research topics, bringing to light the new areas in pediatric dentistry where new research is welcome and highlighting the high and moderate certainty outcomes in the area. In our view, all the reviews included here follow the criteria to be considered a SR: clear clinical question, clear systematic search, reproducible methodology, risk of bias, and systematic presentation of results [6, 10].

There are limitations, such as excluding reviews without the certainty of the evidence to compare quality. Therefore, the reviews analyzed here may not necessarily be fully representative of all pediatric dentistry-related interventions, as we used a convenience sample. The convenience sample of reviews published at a two-year interval limits the external validity. Likewise, our results do not fully represent the Cochrane reviews due to their low number in our sample. Due to the possible overlap of pediatric dentistry and other specialties, there is the possibility of exclusion of SRs covering pediatric dentistry and other areas.

The certainty of the evidence represents the judgment of the review authors, as we did not cross-check their decision. Finally, we did not judge the minimal important difference and magnitude of the effect of the evidence generated.

Perspectives for future research and clinical practice

Future systematic reviews of interventions should require the certainty of the evidence. Furthermore, the authors would benefit from the magnitude of the effect, minimal important difference, and GRADE guideline 26 to interpret the evidence [38, 40]. Our results can help clinicians find high and moderated-quality evidence in pediatric dentistry and help researchers evaluate which low and very low-quality evidence is worthy of research in the future.

Conclusion

Most reviews fulfilled the majority of AMSTAR 2 criteria despite the mainly low and critically low final quality. The proportion of studies assessing the certainty of the evidence through the GRADE approach was small. The main problems of the certainty of the evidence were due to risk of bias and imprecision. Considering that the greatest evidence of these reviews was mainly low and very low, there is space for future clinical trials and SRs in pediatric dentistry.

Registration and protocol

The protocol was registered a priori in the International Prospective Register of Systematic Reviews (PROSPERO #CRD42022365443). There have been no modifications regarding the protocol methods since the start of this overview.

Data availability

All data generated or analyzed during this study are included in this published article as Supplementary Table S4.

Abbreviations

AMSTAR 2:

Assessing the Methodological Quality of Systematic Reviews

CI:

Confidence interval

GRADE:

Grading of Recommendations, Assessment, Development and Evaluation

NRSI:

Non-randomized study of intervention

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RCT:

Randomized controlled trial

SR:

Systematic review

References

  1. Smail-Faugeron V, Fron-Chabouis H, Courson F. Methodological quality and implications for practice of systematic Cochrane reviews in pediatric oral health: a critical assessment. BMC Oral Health. 2014;14:35.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Pandis N, Fleming PS, Worthington H, Salanti G. The quality of the evidence according to GRADE is predominantly low or very low in oral health systematic reviews. PLoS ONE. 2015;10(7):e0131644.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Fleming PS, Koletsi D, Ioannidis JP, Pandis N. High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews. J Clin Epidemiol. 2016;78:34–42.

    Article  PubMed  Google Scholar 

  4. Notaro SQ, Hermont AP, Cruz PV, Maia RM, Avila WM, Poklepovic Pericic T, Abreu LG, Jiao R, Martins-Pfeifer C. Methodological quality of systematic reviews addressing orthodontic interventions: methodological study. Pesq Bras Odontop Clin Int. 2024;24:e230074.

    Article  Google Scholar 

  5. Pereira AG, Martins CC, Campos JR, Faria SF, Notaro SQ, Poklepovic-Pericic T, Costa LC, Costa FO, Cota LO. Critical appraisal of systematic reviews of intervention studies in periodontology using AMSTAR 2 and ROBIS tools. J Clin Exp Dent. 2023;15(8):e678–94.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Pauletto P, Polmann H, Reus JC, de Oliveira JMD, Chaves D, Lehmkuhl K, Massignan C, Stefani CM, Martins CC, Flores-Mir C, De Canto L. G: Critical appraisal of systematic reviews of intervention in dentistry published between 2019–2020 using the AMSTAR 2 tool. Evid Based Dent 2022.

  7. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, Guyatt GH, Harbour RT, Haugh MC, Henry D, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.

    Article  PubMed  Google Scholar 

  8. Mejare IA, Klingberg G, Mowafi FK, Stecksen-Blicks C, Twetman SH, Tranaeus SH. A systematic map of systematic reviews in pediatric dentistry–what do we really know? PLoS ONE. 2015;10(2):e0117537.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ. Welch VAe: Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. www.training.cochrane.org/handbook.

  11. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ. Group GW: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Lai YYL, Zafar S, Leonard HM, Walsh LJ, Downs JA. Oral health education and promotion in special needs children: systematic review and meta-analysis. Oral Dis. 2022;28(1):66–75.

    Article  PubMed  Google Scholar 

  14. Davidovich E, Shafir S, Shay B, Zini A. Plaque removal by a Powered Toothbrush Versus a Manual Toothbrush in children: a systematic review and Meta-analysis. Pediatr Dent. 2020;42(4):280–7.

    PubMed  Google Scholar 

  15. Santos GM, Pacheco RL, Bussadori SK, Santos EM, Riera R, de Oliveira Cruz Latorraca C, Mota P, Benavent Caldas Bellotto EF, Martimbianco ALC. Effectiveness and safety of ozone therapy in Dental Caries Treatment: systematic review and Meta-analysis. J Evid-Based Dent Pract. 2020;20(4):101472.

    Article  PubMed  Google Scholar 

  16. Khan U, MacPherson J, Bezuhly M, Hong P. Comparison of Frenotomy techniques for the treatment of Ankyloglossia in children: a systematic review. Otolaryngol Head Neck Surg. 2020;163(3):428–43.

    Article  PubMed  Google Scholar 

  17. Kamber R, Meyer-Lueckel H, Kloukos D, Tennert C, Wierichs RJ. Efficacy of sealants and bonding materials during fixed orthodontic treatment to prevent enamel demineralization: a systematic review and meta-analysis. Sci Rep. 2021;11(1):16556.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Custódio NB, Costa FS, Cademartori MG, Costa VPP, Goettems ML. Effectiveness of virtual reality glasses as a distraction for children during Dental Care. Pediatr Dent. 2020;42(2):93–102.

    PubMed  Google Scholar 

  19. Manchanda S, Sardana D, Yiu CKY. A systematic review and meta-analysis of randomized clinical trials comparing rotary canal instrumentation techniques with manual instrumentation techniques in primary teeth. Int Endod J. 2020;53(3):333–53.

    Article  CAS  PubMed  Google Scholar 

  20. Xiang B, Wong HM, Perfecto AP, McGrath CPJ. The application of theory-guided oral health interventions in adolescents: a systematic review and meta-analysis of randomized controlled trials. Psychol Health. 2021;36(7):879–94.

    Article  PubMed  Google Scholar 

  21. Chugh VK, Patnana AK, Chugh A, Kumar P, Wadhwa P, Singh S. Clinical differences of hand and rotary instrumentations during biomechanical preparation in primary teeth-A systematic review and meta-analysis. Int J Paediatr Dent. 2021;31(1):131–42.

    Article  PubMed  Google Scholar 

  22. Taneja S, Singh A, Jain A. Anesthetic effectiveness of Articaine and Lidocaine in Pediatric patients during Dental procedures: a systematic review and Meta-analysis. Pediatr Dent. 2020;42(4):273–81.

    PubMed  Google Scholar 

  23. Martins ML, Ribeiro-Lages MB, Masterson D, Magno MB, Cavalcanti YW, Maia LC, Fonseca-Goncalves A. Efficacy of natural antimicrobials derived from phenolic compounds in the control of biofilm in children and adolescents compared to synthetic antimicrobials: a systematic review and meta-analysis. Arch Oral Biol. 2020;118:104844.

    Article  CAS  PubMed  Google Scholar 

  24. Hao S, Ji L, Wang Y. Effect of Honey on Pediatric Radio/Chemotherapy-Induced oral mucositis (R/CIOM): a systematic review and Meta-analysis. Evid-Based Complement Alternat Med; 2022. ID 6906439.

  25. Patnana AK, Chugh VK, Chugh A, Vanga NRV, Kumar P. Effectiveness of zirconia crowns compared with stainless steel crowns in primary posterior teeth rehabilitation: a systematic review and meta-analysis. J Am Dent Assoc. 2022;153(2):158–e166155.

    Article  PubMed  Google Scholar 

  26. Olsson Moller U, Beck I, Ryden L, Malmstrom M. A comprehensive approach to rehabilitation interventions following breast cancer treatment - a systematic review of systematic reviews. BMC Cancer. 2019;19(1):472.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Matthias K, Rissling O, Pieper D, Morche J, Nocon M, Jacobs A, Wegewitz U, Schirm J, Lorenz RC. The methodological quality of systematic reviews on the treatment of adult major depression needs improvement according to AMSTAR 2: a cross-sectional study. Heliyon. 2020;6(9):e04776.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Luo J, Chen Z, Liu D, Li H, He S, Zeng L, Yang M, Liu Z, Xiao X, Zhang L. Methodological quality and reporting quality of COVID-19 living systematic review: a cross-sectional study. BMC Med Res Methodol. 2023;23(1):175.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Gao Y, Cai Y, Yang K, Liu M, Shi S, Chen J, Sun Y, Song F, Zhang J, Tian J. Methodological and reporting quality in non-cochrane systematic review updates could be improved: a comparative study. J Clin Epidemiol. 2020;119:36–46.

    Article  PubMed  Google Scholar 

  30. Wasiak J, Shen AY, Tan HB, Mahar R, Kan G, Khoo WR, Faggion CM Jr. Methodological quality assessment of paper-based systematic reviews published in oral health. Clin Oral Investig. 2016;20(3):399–431.

    Article  CAS  PubMed  Google Scholar 

  31. Lasserson TJ, Thomas J, Higgins JPT, Toby J, Lasserson J, Thomas. Julian PT Higgins. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Chapter 1. www.training.cochrane.org/handbook.

  32. Papageorgiou SN, Dimitraki D, Coolidge T, Kotsanos N. Publication bias & small-study effects in pediatric dentistry meta-analyses. J Evid-Based Dent Pract. 2015;15(1):8–24.

    Article  PubMed  Google Scholar 

  33. Zhang Y, Akl EA, Schunemann HJ. Using systematic reviews in guideline development: the GRADE approach. Res Synth Methods. 2018;10(3):312–29.

    Article  Google Scholar 

  34. Ahn RWA, Abraham A, Saba S, Korenstein D, Madden E, Boscardin WJ, Keyhani S. Financial ties of principal investigators and randomized controlled trial outcomes: cross sectional study. BMJ. 2017;356:i6770.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Amiri AR, Kanesalingam K, Cro S, Casey AT. Does source of funding and conflict of interest influence the outcome and quality of spinal research? J Spine 2014, 14(2):308–14.

  36. Martins CC, Riva JJ, Firmino RT, Colunga-Lozano LE, Granville-Garcia AF, Zhang Y, Schünemann HJ. Conflict of interest is not associated with positive conclusions in toothpaste trials: a systematic survey. J Clin Epidemiol. 2019;108:140–6.

    Article  Google Scholar 

  37. Bassani R, Pereira GKR, Page MJ, Tricco AC, Moher D, Sarkis-Onofre R. Systematic reviews in dentistry: current status, epidemiological and reporting characteristics. J Dent. 2019;82:71–84.

    Article  PubMed  Google Scholar 

  38. Carrasco-Labra A, Devji T, Qasim A, Phillips MR, Wang Y, Johnston BC, Devasenapathy N, Zeraatkar D, Bhatt M, Jin X, et al. Minimal important difference estimates for patient-reported outcomes: a systematic survey. J Clin Epidemiol. 2021;133:61–71.

    Article  PubMed  Google Scholar 

  39. Schunemann HJ, Neumann I, Hultcrantz M, Brignardello-Petersen R, Zeng L, Murad MH, Izcovich A, Morgano GP, Baldeh T, Santesso N, et al. GRADE guidance 35: update on rating imprecision for assessing contextualized certainty of evidence and making decisions. J Clin Epidemiol. 2022;150:225–42.

    Article  PubMed  Google Scholar 

  40. Santesso N, Glenton C, Dahm P, Garner P, Akl E, Alper B, Brignardello-Petersen R, Carrasco-Labra A, De Beer H, Hultcrantz M, et al. GRADE guidelines 26: informative statements to communicate the findings of systematic reviews of interventions. J Clin Epidemiol. 2019;S0895–4356(19):30416–30410.

    Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

RAB, JRC and AGP received a scholarship from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES, Ministry of Education, Brazil; grant #001). SQN and ACSO received a scholarship from the Pró-Reitoria de Pesquisa da Universidade Federal de Minas Gerais (PICIC/ PIBIT/ PRPq/ CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico, Ministry of Education, Brazil); GDLC is research fellow at Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq, Ministry of Education, Brazil). LPS received a scholarship from Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG #APQ-00323-17). FAPEMIG funded the study (#APQ-00323-17). The funding agencies had no role in the conception of the research and interpretation of data.

Author information

Authors and Affiliations

  1. Department of Surgery, Clinical Dentistry and Oral Pathology and Oral Surgery, Dental School, Federal University of Minas Gerais, Belo Horizonte, Brazil

    Rachel Alvarenga-Brant, Alexandre Godinho Pereira & Julya Ribeiro Campos

  2. Department of Pediatric Dentistry, Dental School, Federal University of Minas Gerais, Belo Horizonte, Brazil

    Sarah Queiroz Notaro, Luciana Póvoa-Santos, Ana Clara Souza-Oliveira & Carolina Castro Martins-Pfeifer

  3. Department of Dentistry, University of Brasília, Brasília, DF, Brazil

    Cristine Miron Stefani

  4. Department of Dentistry, Federal University of Santa Catarina, Florianópolis, Brazil

    Graziela De Luca Canto

Authors
  1. Rachel Alvarenga-Brant
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sarah Queiroz Notaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cristine Miron Stefani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Graziela De Luca Canto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alexandre Godinho Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Luciana Póvoa-Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ana Clara Souza-Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Julya Ribeiro Campos
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Carolina Castro Martins-Pfeifer
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SQN and CCMP: conception, design, writing, and editing. RAB: data extraction (GRADE approach), data analysis, writing and editing. SQN, ACSO, LPS, JRC, AGP: selection of studies, data extraction and methodological quality. SQN: design of search strategy, electronic search. CMS, GDLC and CCMP: final review and editing. All authors read and approved the final version of the manuscript.

Corresponding author

Correspondence to Carolina Castro Martins-Pfeifer.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alvarenga-Brant, R., Notaro, S.Q., Stefani, C.M. et al. Pediatric dentistry systematic reviews using the GRADE approach: methodological study. BMC Oral Health 24, 787 (2024). https://doi.org/10.1186/s12903-024-04542-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12903-024-04542-w

Keywords

BMC Oral Health

ISSN: 1472-6831

Contact us