Developing the teledentistry acceptance survey for dentists – TAS-D: a Delphi study

Abstract

Introduction

The increasing interest in teledentistry since the COVID-19 pandemic warrants an evaluation of dentists’ willingness to adopt it. This study aimed to develop a questionnaire to assess dentist’s intention to use teledentistry and the associated factors.

Methods

A literature search was used to identify items for the questionnaire. The Unified Theory of Acceptance and Use of Technology (UTAUT2) was adopted as framework. A Delphi panel was constituted of researchers with relevant publications and the International Association of Dental Research e-Oral Health Network members. Three Delphi consultations were conducted to establish consensus on items. Consensus was set at 80% agreement and content validity ratio (CVR), reaffirmed iteratively.

Results

Nineteen out of 25 (76%) invited experts participated in the first round, 17 in the second and 15 in the third. The preliminary questionnaire had 81 items in three sections, reduced to 66, 45 and 33 items in the first, second and third rounds. After revision, the final version comprised eight items assessing dentists’ backgrounds in Sect. 1, seven items identifying teledentistry uses in Sect. 2, and 17 items assessing intention to use teledentistry and its determinants in seven dimensions in Sect. 3. The initial CVR was 0.45, which increased to 0.80 at the end of the third round.

Conclusion

A survey tool was developed to assess the acceptance of teledentistry, and its determinants based on the UTAUT2 framework through consensus among teledentistry experts. The tool had excellent validity and needs further evaluation of its psychometric properties.

Introduction

Restricted movement during COVID-19 was the driving force behind the large-scale use of information and communication technology (ICT) to enable the continuation of business, education, healthcare, and other activities [1]. The abrupt change caused by the pandemic facilitated the use of ICT applications that have been previously proposed but were poorly implemented. Teledentistry is an ICT application that has received sustained interest and a significant surge in use after the COVID-19 pandemic [2, 3]. Evidence attests to the effectiveness of teledentistry in enabling remote consultations, disseminating and following up preventive oral health measures, and improving oral healthcare delivery during emergencies and natural disasters [4].

The adoption of teledentistry, however, varies across countries and settings and is influenced by political support, legal frameworks, ICT infrastructure, possibility of billing, oral health professionals’ teledentistry training and exposure, teledentistry acceptance by patients and dentists, to name a few [3]. Evidence suggests that decision-makers have accepted the use of teledentistry in several countries [5]. However, the adoption of its use is largely dependent on dentists’ acceptance and their willingness to invest time and money for a wider scale implementation of teledentistry, among other factors [6]. A comprehensive assessment of dentists’ acceptance of teledentistry is needed across sectors and settings. However, teledentistry acceptance has been assessed using multiple non-validated instruments, limiting meaningful comparisons of findings [7]. Also, little is known about the factors influencing dentists’ willingness to adopt teledentistry. Yet, this information is important to promote the institution of enabling factors and address the barriers to the adoption of teledentistry.

The Unified Theory of Acceptance and Use of Technology (UTAUT) provides a theoretical framework for examining the factors influencing the acceptance of ICT [8]. This theory explains ICT adoption through four constructs: performance expectancy (the anticipated benefits of using ICT applications or systems), effort expectancy (the perceived ease of use), social influence (the impact of others’ opinions on user’s decision to adopt ICT) and facilitating conditions (the availability of necessary infrastructure and support for ICT utilization). To address certain limitations of the original UTAUT model, an enhanced version, UTAUT2, was developed. This iteration introduces additional constructs tailored to the context of emerging technologies and a diverse user base [9]. These constructs include hedonic motivation (the enjoyment derived from using the ICT application or system), price value (the cost-benefit assessment of using the ICT), and habit and experience (users’ familiarity with the ICT system) [9].

The UTAUT2 framework has been applied in various contexts, such as developing an eHealth acceptance scale in France [10], evaluating patients’ acceptance of telemedicine in a cross-country survey [11], and appraising medical students’ acceptance of blended learning [12]. Hence, the study aimed to develop a questionnaire to assess dentists’ acceptance of teledentistry and identify the factors influencing this acceptance based on the UTAUT2 framework.

Methods

Ethical approval was granted by the Research Ethics Committee at the Faculty of Dentistry, Alexandria University, Egypt (0575-01/2023). Written informed consent was obtained from the experts participating in the study when they confirmed their agreement to participate by email after receiving the invitation to collaborate.

Study design

A three-round Delphi consensus-building process [13] was initiated involving a panel of experts specializing in e-oral health. The experts were asked to evaluate a comprehensive set of items organized under the constructs of the UTAUT2. This study serves as the inaugural phase of a project that seeks to develop and validate a tool to assess oral health professionals’ and patients’ acceptance of teledentistry.

Rationale for the Delphi process

The Delphi technique is a consensus-building method that solicits anonymous and independent stakeholder feedback regarding a specific question. This approach enables stakeholders with varying levels of expertise to provide feedback without succumbing to peer pressure [14]. The Delphi technique can be efficiently conducted online and is well-suited for building consensus among experts from different countries or geographic regions. The process involves developing an initial questionnaire, then distributing it to stakeholders to rate its relevance to the research question and offer comments. Ratings are used to discriminate between relevant and irrelevant items to filter items, while comments suggest modifications. Subsequently, an updated version of the questionnaire undergoes another round of rating and commenting in an iterative process that persists until a consensus is achieved [13]. We adhered to the CREDES guidelines for conducting and reporting Delphi studies [15].

Identification of candidate items for the survey tool

From February to March 2023, we conducted a thorough and systematic literature search to identify studies employing the UTAUT2 framework to assess the acceptability of ICT applications in healthcare. PubMed database was searched using the terms “information technology” or “ICT,” combined with “technology acceptance” or “technology adoption” or “technology use” AND “UTAUT” or “UTAUT2” or “Unified Theory of Acceptance and Use of Technology.” Additionally, these terms were combined with terms such as “teledentistry” or “telemedicine” or “e-health” or “m-health” or “e-oral health” or “m-oral health” or “mobile health” or “mobile oral health” or “digital health.” Two independent reviewers (MET and NA) screened the titles and abstracts of the resulting citations to identify relevant studies. Full-text articles of the selected studies were then reviewed to confirm their use of UTAUT2 in the context of ICT acceptance in healthcare. Any discrepancies were resolved through discussion with a third reviewer (MOF). We searched PubMed, Scopus and Google Scholar in addition to hand searching relevant journals and the reference lists of the eligible studies to identify missing publications. We extracted relevant questions and their response options.

The preliminary version of the questionnaire was based on the structure used by Hayotte et al. [10], then we added further items identified from additional sources under the UTAUT2 dimensions [16,17,18,19,20,21,22,23,24,25,26]. Items were added until saturation occurred, and further items provided no new concept. The questionnaire was developed in English and the preliminary version is shown in Appendix 1. All items were included in consensus between the core study team members.

The questionnaire comprised three sections similar to what was used in previous studies [10, 16,17,18,19,20,21,22,23,24,25,26]. The first section had nine questions assessing dentists’ background namely gender, age, country of residence, highest academic degree, number of years practicing dentistry, whether the respondent was a general dental practitioner or a specialist, which specialty, number of patients per day, and workplace characteristics.

The second section assessed whether respondents agree that teledentistry would help in conducting nine procedures namely remote assessment, video consultations, emergency advice, online development of a treatment plan, explaining a treatment plan, providing dental hygiene education, training patients for dental hygiene, patient monitoring and sending patient information to others. Each question had a ‘yes/no’ response.

The third section assessed the facilitators and barriers to teledentistry. This section had 63 items that measured 10 dimensions based on the seven constructs of the UTAUT2 framework [9]. These dimensions were Performance Expectancy (PE) divided into non-patient centered (PEN) and patient centered (PEP) assessing the benefits of teledentistry to dentist/ practice and to patients, acknowledging perceived utility as a primary driver of adoption, Effort Expectancy (EE) assessing the perceived ease of using teledentistry since users are more likely to embrace technology they perceive as easy to use, Social Influence divided into General (SI-G), Organizational (SI-O), and Patients (SI-P) highlighting the role of subjective norms and the influence of significant others in different circles in shaping users’ decision regarding teledentistry, Facilitating Conditions (FC) addressing the adequacy of infrastructure and support systems on adopting teledentistry, Hedonic Motivation (HM) and Price Value (PV) considering the emotional and economic aspects, in terms of enjoyment and affordability in shaping user attitudes and Experience and Habit (EH) acknowledging the role of dentist’s experience in facilitating the use of new technology. The section also had an 11th dimension assessing behavioral intention (BI) to use teledentistry. Participants indicated their agreement on a scale from 1 (Strongly Disagree), 2 (Disagree), 3 (Neither Agree nor Disagree), 4 (Agree) to 5 (Strongly Agree).

Identification of experts

Recognizing the distinct roles of dentists and patients in teledentistry acceptance, this study focused on developing the Teledentistry Acceptance Survey for Dentists (TAS-D), as separate tools are necessary to address the unique perspectives of each group.

The expert panel comprised researchers engaged in e-oral health research, who had published at least one manuscript on teledentistry in the last five years. Invitations were also extended to members of the International Association of Dental Research (IADR) e-Oral Health Network (eOHN). We aimed to have geographic representation because technology infrastructure and legislative frameworks affect dentists’ acceptance and use of teledentistry, and these differ by region. Also, involving experts from different regions would enhance the generalizability of the proposed tool. We aimed to include 15 to 30 panel members. A larger group may reduce the response rate and make group management difficult without better outcomes [27].

Invitations were emailed to potential panel members by the Principal Investigator (PI), copying in the study’s core team. The email outlined the study purpose and requested interested experts to reply, indicating their consent to participate. Invitations were extended to 42 experts, including 21 females (50%), from North America (9), South America (5), Australia (6), Europe (9), Asia (5), and Africa (8). Ten (23.8%) experts did not respond, 7 (16.7%) declined participation, and 25 (59.5%) agreed to join the Delphi panel.

Delphi rounds

The consensus-building process proceeded in three rounds [15], with experts receiving instructions and a link to an electronic validation form via email from the PI in each round. The anonymity and objectivity of the process were maintained and only the PI knew the voting patterns on the items and who the voters were.

The initial version of the questionnaire was uploaded to SurveyMonkey, featuring a rating scale for experts to express the relevance of each item. The scale ranged from 1 (least relevance) to 5 (highest relevance). Experts could also suggest modifications in a comment box after each section.

In the first round, which began on January 18th, 2023, and lasted three weeks, experts were tasked with evaluating items to streamline their quantity, retain the most relevant, select optimal alternatives, verify alignment with specified dimensions, and provide feedback.

The second round commenced on April 9th, 2023, and closed three weeks later. Instructions provided an overview of the changes made after the first round, urging experts to retain only relevant items, remove duplicates, and verify correct categorization under each dimension.

The third and final round began on June 1st, 2023, and lasted three weeks. Instructions emphasized that this round did not include any additions or changes to item dimensions from previous rounds. Experts were instructed to categorize items as essential or non-essential, minimize neutral responses, and refine item wording for clarity.

Definition of consensus and analysis

The data were retrieved as an Excel file (Microsoft Corp., WA, USA) from SurveyMonkey, and the items underwent both quantitative and qualitative evaluation against predefined criteria. For the quantitative evaluation, we recoded the relevance scores 1 or 2 as non-essential and relevance scores 4 or 5 as essential.

We followed a 4-step sequential process to build consensus. The first two steps were based on the percent agreement of the recoded relevance rating. In the first step, items labelled as non-essential by more than 80% of experts were removed. In the second step, items identified as essential by at least 80% of experts were retained [28, 29]. The next two steps were based on the content validity ratio (CVR) calculated as CVR = [n_e − (N/2)] / (N/2), where n_e is the number of experts indicating that the item was essential, and N is the total number of experts [30]. The CVR represents the degree to which the items reflect the content of its dimension. The CVR ranges from 1 to -1, with higher values indicating greater agreement on the item’s relevance to the questionnaire [31]. In the third step, items with negative CVR were removed signifying that less than 50% of experts considered them essential [30]. In the fourth step, the items were evaluated against the revised critical values of Lawshe’s CVR that varied by the number of experts in the panel. Based on this, we retained items with CVR equal or greater to the critical value [32].

Calculations were performed after each round to refine the questionnaire for subsequent rounds. Items that did not meet the inclusion criteria were excluded. The CVR was assessed for improvement and recalculated with new expert ratings in the next round. Items that introduced unique concepts and had more neutral than negative votes were retained for further review. Qualitative adjustments based on expert feedback included rephrasing, repositioning, or adding items. The process was iterated until all items reached essential status with positive CVRs. A core team member not involved in the Delphi process conducted a final review and made minor changes to the questionnaire.

Results

In the first round, 19 (76%) participated of 25 experts who initially consented to join the panel. Table 1 shows that the experts were geographically diverse, with three representatives maximum from any country. Participation was skewed towards females, professionals with direct clinical care duties, experts with a Ph.D. degree, and aged 45 years and older. Seventeen (89.5%) experts participated in round 2, and 15 (79%) participated in round 3. An overview of the process is illustrated in Fig. 1.

Table 1 The profiles of experts participating in the Delphi panel (n = 19)

Fig. 1

The Delphi process rounds

Round 1

All items in Sect. 1 were considered essential and no modifications were suggested. For Sect. 2, a new item was added: “Assisting with referrals”, another was rephrased to “Screening patient’s oral condition remotely”, and the response scale was changed from yes/no to a 5-point Likert scale ranging from “not at all” to “very much”.

For Sect. 3, three items were labelled as non-essential by more than 80% of experts and were excluded. Seven items were identified as essential by at least 80% and were retained. Twenty-two items had negative CVR and were removed. For a panel of 19 experts, the revised critical value for Lawshe’s CVR was 0.474. A total of 15 items were above this cutoff point and were retained. Twenty-three items below this cutoff point were retained for further assessment in round 2 since there were more undecided than non-essential votes for them. Also, 26 items were rephrased, and 10 items were shifted across dimensions. The label of the dimension Price Value (PV) was changed to “Perceived Price Value” (PPV). Additionally, two new items were added: “Referral using teledentistry provides specialists with more complete picture of patient’s condition before examining the patient” to the Performance Expectancy - non-patient centered (PEN) dimension, and “Teledentistry easily fits within my clinical workflow” was added to the Effort Expectancy (EE) dimension.

After round 1, there were 9 items in Sects. 1, 10 items in Sect. 2, and 47 items in 10 dimensions in Sect. 3, after removing one dimension (Experience and Habit), with a total of 66 items in the questionnaire. The CVR ranged from 0.14 to 0.90, with an overall hypothetical mean CVR after modifications = 0.45 indicating improvement from the initial CVR = 0.21 (Appendix 2).

Round 2

No changes were made to Sect. 1 in round 2. For Sect. 2, one item, “training patients for oral hygiene”, was rated as essential by 64.7% of the experts and its CVR was 0.294, lower than the cutoff point of CVR ≥ 0.529 for 17 responding experts in this round. Thus, this item was removed leaving 9 items in Sect. 2. For Sect. 3, none of the 47 items were considered non-essential, although 8 (17%) items had negative CVR and were removed. Thirteen (27.7%) items were considered essential, and 5 more items had CVR equal or greater than the critical value. Thus, 21 items from this section qualified for inclusion in round 3, to which 6 items were added for further testing. After round 2, Sect. 1 included 9 items, Sect. 2 included 9 items and Sect. 3 included 27 items in 9 dimensions after removing one dimension (Social Influence - Organizational), with a total of 45 items. The CVR was 0.44 and after removing the specified items, the hypothetical CVR increased to 0.64 (Appendix 3).

Round 3

No changes were made to Sect. 1. At least 80% of the experts agreed that 7 of 9 items in Sect. 2, and 17 of 27 items in Sect. 3 were essential. None of the items in any section had negative CVR and, therefore, no item was removed because of this reason. Fifteen experts responded to this round and the critical CVR was 0.600. None of the items, other than those voted as essential by at least 80% of experts, had CVR more than this critical value so the remaining items were removed. Eventually, Sect. 1 included 9 items, Sect. 2 included 7 items and Sect. 3 included 17 items in 7 dimensions after removing two dimensions (Social Influence – General and Social Influence – Patients), with a total of 33 items. The CVR was 0.66 and increased to 0.80 after removing the non-eligible items (Appendix 4). Table 2 offers an overview of the modifications to the items.

Table 2 Summary of questionnaire item modifications informed by experts’ feedback

In the revision of the final version of the questionnaire, minor edits were made including merging two questions in Sect. 1 that asked about dentists’ specialties, thus, reducing the number of items in Sects. 1 to 8 instead of 9. In Sect. 2, the 5-point Likert scale was replaced with a scale ranging from zero “does not help at all” to 10 “extremely helpful”. In Sect. 3, minor rephrasing of the last item was made. Appendix 5 shows the final questionnaire of 32 items.

Discussion

We used a Delphi process to develop the TAS-D. After three Delphi rounds involving 15 to 19 experts who added, modified, shuffled, and removed items, and a final review by a study core team-member, the final version of TAS-D included 32 items: 8 in Sects. 1, 7 in Sect. 2, and 17 in Sect. 3. Section 3 explored the barriers to using teledentistry in seven dimensions representing five UTAUT2 constructs; with an 8th dimension assessing behavioral intentions. The CVR of the final questionnaire was 0.80. The reduced number of items and dimensions with high CVR provide a questionnaire that can assess dentists’ acceptance of teledentistry pending the assessment of its psychometric properties in future studies.

Several important findings emerged. First, the study gives insights about the scope of teledentistry based on feedback from experts. The modifications of items in Sect. 2 indicated that the experts favored the restriction of teledentistry to verbal communication such as advice, care of emergencies, consultations and explanations, follow up and referral and remote screening. Procedures like treatment planning or oral hygiene training or education were excluded. This consensus represents an approach to teledentistry that can allow care provision at times like the COVID-19 pandemic [3] and address more than 70% of the needs of individuals with oral health complaints [33]. Conversely, dental surgeons in China demonstrated that teledentistry can support hands-on procedures like the remote placement of an implant using robots controlled by 5G networks. However, the infrastructure allowing this may be well beyond reach in most settings [34]. Thus, the range of procedures included in this survey represents the most realistic and impactful scope of teledentistry.

Second, the study provides a perspective on the factors that experts considered unrelated to teledentistry acceptance. The Delphi rounds removed the Social Influence and Experience and Habit dimensions, indicating agreement that teledentistry should not be influenced by dentists’ personal or professional social circles or their habits and experience. This may be attributed to the individualistic nature of dental practice, the professional autonomy that dentists enjoy [35] and the emphasis on evidence-based practice in decision-making [36]. On the other hand, subjective norms may have a stronger influence on mandatory than voluntary behaviors [37] and may represent a social pressure to resist [20]. Social Influence might have been removed because the experts felt that neither the profession nor the public may have – yet – formed an opinion to drive Social Influence [16], and that insufficient time has passed to build dentists’ Experience and Habit. This agrees with research showing that Social Influence was neither significantly associated with medical professionals’ acceptance of artificial intelligence [16], Filipino physiatrists’ acceptance of tele-rehabilitation [17], nor Korean nursing students’ attitudes toward using artificial intelligence [25]. By contrast, a study showed a significant association between Social Influence and nurses’ intention to use mobile learning, which was mandated by their hospital [22]. We postulate that the exclusion of the dimensions measuring Social Influence and Experience and Habit from the TAS-D may reflect the current early stage of teledentistry use. Therefore, contextual factors such as organizational mandates and supportive environments may influence teledentistry use [22].

Third, Performance Expectancy had the highest CVR reflecting experts’ agreement on teledentistry’ s benefits to patients including care provision in remote settings and timely care, both of which have been tested during the COVID-19 pandemic [2]. Teledentistry also has the potential to reduce the carbon footprint of dental practice by decreasing patients’ travel to the clinic with implications for sustainability, conservation of energy and reduced pollution [3, 38].

Fourth, Perceived Price Value, which had a high CVR, linked the use of teledentistry to insurance coverage. Funding and remuneration plans play crucial roles in facilitating the adoption of teledentistry and transitioning from research-oriented applications to broader implementation within the healthcare systems [3]. Professional dental bodies need to advocate for the incorporation of teledentistry within health systems and insurance plans coverage so that remote services can be available to patients.

Fifth, Hedonic Motivation had the third highest CVR. Hedonic Motivation is related to how interesting dentists perceive teledentistry. The novelty effect could influence dentists’ initial acceptance of teledentistry in its early stages [39]. However, as time progresses, acceptance driven by novelty is anticipated to give way to a more realistic evaluation that must be accompanied by a supportive legal/ regulatory framework of the scope of practice and financial arrangements funding the service [40]. As the digital natives of Generation Z, born between 1996 and 2012, transition from studying to practicing dentistry [41] it may be assumed that more dentists will use technology because they enjoy it.

The study had some limitations. Most experts were senior professionals, which may have influenced the responses. We aimed to minimize selection bias by inviting males and females and experts from various regions to increase generalizability. The study’s strengths include using the Delphi consensus building method to ensure that all experts provided their input with the least bias due to social desirability or the impact of power dynamics among the panel members. The panel was formed by purposefully selecting experts and potential items covering a wide range of scenarios. Furthermore, most experts regularly engage in direct clinical care duties, demonstrating a profound understanding of the practical realities of patient management and an understanding of the challenges associated with introducing new technologies into clinical practice. The study holds implications for defining the scope of teledentistry and understanding the factors influencing its acceptance among dentists. Future studies are needed to assess the validity of the proposed tool among other oral healthcare professionals and its psychometric properties among dentists.

The development of the preliminary version of TAS-D is a step toward the standardized assessment of dentists’ acceptance of teledentistry. This allows valid comparisons across dental settings, practitioners and technologies before teledentistry can be integrated into practice. For example, funders supporting research or implementation projects for teledentistry would appreciate knowing the level to which the planned intervention would be accepted by dentists who will use it. Also, policy makers instituting changes in healthcare systems need to understand the acceptance or resistance to the use of teledentistry before embarking on such a campaign. Dental schools and healthcare facilities investing in training and infrastructure to support and enable the incorporation of teledentistry into practice should also assess the existing attitude toward this technology and identify whether interventions are needed to promote positive professional attitude.

This study marks the initial phase in assessing dentists’ acceptance of teledentistry, offering a standardized and comprehensive approach for comparisons across subgroups. Such an approach facilitates the identification of barriers to wide scale teledentistry implementation and developing and assessing interventions to drive implementation efforts.

Conclusion

The TAS-D assesses dentists’ profile, their opinions regarding the uses of teledentistry, intention to use teledentistry, and its determinants. The experts removed items in two dimensions of the UTAUT2, social influence, and experience and habits. The questionnaire had a high CVR indicating good agreement among experts on item relevance. This study opens the opportunity to assess the acceptance of teledentistry using a validated tool, thereby allowing for comparison of results. Future studies are needed to assess the psychometric properties of the survey and evaluate it among oral healthcare professionals.