Skip to main content
Download PDF

Assessing the usability and reliability of a web-based teledentistry tool for remote diagnosis of oral lesions: a cross-sectional study

BMC Oral Health volume 24, Article number: 1094 (2024) Cite this article

Abstract

Background

Oral mucosa lesions are the third most prevalent oral pathology, following caries and periodontal diseases. Teledentistry offers an effective way to manage patients with these lesions. The accuracy of remote diagnoses and consultations relies heavily on the quality of the information and photos sent to remote specialists. This study aims to evaluate the usability and reliability of a teledentistry tool for the remote diagnosis of oral lesions.

Methods

The cross-sectional study included both usability evaluation and reliability assessment. The teledentistry platform, "OralMedTeledent", facilitated synchronous and asynchronous interactions, allowing for patient consultations, remote follow-ups, and doctor-to-doctor consultations. Usability was evaluated by 5 experts using the Nielsen heuristic checklist. Reliability was assessed from August 2022 to September 2023 with 109 patients, using Cohen's kappa coefficient to measure agreement between examiners and the gold standard in diagnosing oral lesions.

Results

The findings revealed 66 usability issues, most of which were related to helping users recognize, diagnose, and recover from errors, as well as issues with help and documentation. Among these, 11 issues were of minor severity. The reliability test, conducted with 109 participants (57.8% female, 42.2% male) showed that the web-based teleconsultation system performed significantly well. The system demonstrated significant substantial performance (0.81 ≤ κ < 1; P > 0.05).

Conclusion

Overall, the web-based teleconsultation system has proven to be reliable for the remote diagnosis of oral lesions, making it a valuable alternative during emergencies such as the COVID-19 pandemic. However, several usability issues have been identified and need to be addressed.

Peer Review reports

Background

Oral mucosa lesions are the third most prevalent oral pathology, following caries and periodontal diseases [1]. These lesions refer to abnormalities or changes in the tissues lining the oral cavity [2, 3]. They can vary in size, appearance, and location, and may be caused by infection, inflammation, trauma, or neoplastic processes [4]. The overlapping signs and symptoms of these conditions present significant diagnostic challenges, which can only be resolved through clinicopathological tests and the expertise of an experienced oral medicine specialist [5]. Therefore, proper classification, diagnosis, and management of oral mucosal lesions are essential for providing appropriate care and ensuring positive patient outcomes [5, 6].

Teledentistry offers an effective approach to managing patients with oral lesions [7]. It involves the use of electronic health records, digital imaging, photography, and information and communications technology to transmit clinical information over the Internet both synchronously and asynchronously [8]. Teledentistry enables dental professionals to collaborate by exchanging insights and expertise regarding challenging cases, uncertainties, and patient care [9, 10]. This approach allows general practitioners to quickly access specialist opinions, ensuring optimal patient outcomes [8, 11]. Additionally, applying teledentistry in the field of oral medicine field has several advantages, including reducing unnecessary referrals [12,13,14], improving the quality of care [14, 15], facilitating patient-professional communication [14, 16, 17], enabling continuous virtual consultations and follow-ups [18,19,20], and monitoring oral medical emergencies [16, 19].

Telediagnosis and teleconsultation of oral mucosa lesions are among the most widely used applications of teledentistry in oral medicine. [9]. Several studies have highlighted teledentistry tools, including using smartphones to photograph oral lesions [21,22,23,24,25,26]. These photos, along with other clinical information, are shared through various means such as social media platforms [10, 23, 27, 28], emails [29], and cloud-based teledentistry platforms [14, 21, 30]. According to some studies, the accuracy of remote diagnoses and consultations depends on the quality of the information and photos sent to remote specialists [14, 31,32,33]. Therefore, testing the reliability of teledentistry tools is crucial, as emphasized in many studies [31, 34, 35].

In addition to testing the reliability of teledentistry platform, usability is a key consideration in the design and evaluation of software, websites, applications, and various other products to ensure they meet the needs and expectations of their users [36]. Usability refers to the extent to which a product, system, or interface can be used by a specified set of users to achieve specific goals with effectiveness, efficiency, and satisfaction in a particular context [37]. Assessing usability allows for the identification and resolution of difficulties, ensuring that telehealth systems are beneficial. Usability assessment covers aspects such as ease of learning, retention of task performance over time, speed of task performance, a low error rate, and subjective user satisfaction [38, 39]. This study aims to evaluate the usability and reliability of a web-based tele dentistry tool for diagnosing oral lesions through a cross-sectional investigation.

Methods

General description of teledentistry platform

The teledentistry platform, "OralMedTeledent," is a web-based teleconsultation system specifically designed for the remote diagnosis of patients with oral lesions. This system supports both synchronous and asynchronous interactions, allowing for consultations between patients and oral medicine specialists, remote follow-ups of patient conditions, doctor-to-doctor consultations, recording and sending photographs of oral cavity lesions to remote consultants, and the transmission of demographic and clinical information for remote consultations.

Evaluation method

This cross-sectional study included two phases: usability evaluation and reliability assessment. The Overview of research method is illustrated in Fig. 1.

Fig. 1
figure 1

The overview of research method

Full size image

Phase one: usability evaluation

In this phase, usability was assessed using the Nielsen heuristic checklist, which includes ten principles: 1) visibility of system status; 2) match between system and the real world; 3) user control and freedom; 4) consistency and standards; 5) error prevention; 6) recognition rather than recall; 7) flexibility and efficiency of use; 8) aesthetic and minimalist design; 9) help users recognize, diagnose, and recover from errors; and 10) help and documentation. Heuristic evaluation is a widely used method for assessing usability, where 3 to 5 usability experts review the user interface based on these established principles [40].

A team of five specialists with expertise in heuristic evaluation conducted the usability assessment. The team included a PhD in Health Information Management, a PhD in Medical Informatics, a software engineer with experience in various health information systems, an M.Sc. student in Health Information Technology familiar with health information systems, and a professor in the Oral Medicine field. Each evaluator had previous experience with usability evaluation studies, ensuring their familiarity with heuristic evaluation methods.

The evaluators individually reviewed the teleconsultation system's user interface. They marked 'yes' for usability items that were met and 'no' for those that were not. For items marked "no", they rated the severity of the problem on a scale from 0 to 4 using Nielson`s scale (Table 1). The collected data were then analyzed using descriptive statistics in an Excel sheet.

Table 1 Nielsen’s severity rating scale for usability problem
Full size table

Phase two: reliability assessment

Ethical consideration

A cross‐sectional study was conducted from August 2022 to September 2023 with a convenience sample of patients who visited the Oral, Maxillofacial and Dental Diseases Diagnosis Department at Shiraz Faculty of Dentistry in Iran. The study protocol was approved by the Research Ethics Committee of Shiraz University of Medical Science, Iran, under code IR.SUMS.DENTAL.REC.1401.072.

Before commencing the study, participants were thoroughly informed about the study`s objectives and conditions to ensure ethical considerations and awareness. They were made aware that participation was entirely voluntary and would not affect their treatment process. After being fully informed, participants provided written consent.

Patient confidentiality was strictly maintained throughout the study. Only the treatment team had access to patient information. To protect patients' privacy, participants were enrolled in the teledentistry platform anonymously, identified only by unique codes like 'patient 38'. During oral lesion photography, only the oral cavity and lesions were captured to avoid including any identifiable features such as the face. Images of the forehead or eyes were deliberately excluded.

To safeguard patient digital data, security measures were implemented, including user access controls, authentication procedures with usernames and passwords, and the use of secure servers managed by Shiraz University of Medical Sciences. Data encryption protocols were applied during both transmission and storage.

Patients did not participate directly in teledentistry interventions; only photographs of their oral lesions were used to assess the agreement between in-person and remote diagnoses.

Sample size and inclusion criteria

The required sample size was estimated to be 97 participants based on statistical data from similar studies [21, 34]. The calculation was performed using the Kappa coefficient method with the online Sample Size Calculator tool (https://wnarifin.github.io/ssc_web.html) [41, 42]. Details of the sample size calculation method and formula are given below. To account for potential dropouts or loss to follow-up, a total of 109 samples were ultimately collected for the study.

The method of calculating the sample size based on the kappa coefficient:

  • Kappa (2 raters)—estimation2

    • Expected kappa (K): 0.66

    • Precision (± expected): 0.15

    • Proportion of outcome (p): 0.5

    • Confidence level 100(1-α): 95%

    • Expected drop-out rate: 10%

    • Sample size: n = 97

    • Sample size (with 10% drop-out), n drop = 108

The inclusion criteria for this study required patients to have been diagnosed with oral lesions, including white and red lesions, ulcerative, vesicular, and bullous lesions, oral exophytic lesions, hard tissue lesions, and pigmented lesions of the oral mucosa. Patients with conditions unrelated to oral lesions, such as orofacial pain disorders, were excluded from this study.

Study design

This study comprised four steps:

  • I. Step 1: Physical examination

Eligible patients were examined by a professor (examiner 1) from the Oral, Maxillofacial, and Dental Diseases Department. Each patient was then assigned to a code, and their clinical information and final diagnosis were documented.

  • II. Step 2: Photography of oral lesions in patients

After the patient was examined, two residents trained in oral cavity photography took pictures of the patient's mouth, focusing on the oral lesions. The photos were taken with an iPhone featuring a 10-megapixel camera and a display resolution of 720 × 1280 pixels with autofocus (Fig. 2). In total, 262 photos were captured by two residents, who then reviewed the photographs for clarity and to ensure all relevant areas were covered.

  • III. Step 3: Data Entry into the web-based teleconsultation system

Fig. 2
figure 2

Example of an oral lesion with anterior and lateral views captured using an iPhone 13 Pro Max. Case number 38, final diagnosis: Oral lichen planus

Full size image

Initially, two residents and two professors were trained on how to use the web-based teleconsultation system. The residents then submitted patient information, including photographs of oral lesions, demographic data, and clinical details, as separate teleconsultation requests through the application, without including the final diagnosis. These requests were sent to two professors in the department, who served as examiners (Examiner 1 and Examiner 2). The professors had profiles on this web-based teleconsultation system, which allowed them to access and respond to the consultation requests.

  • IV. Step 4: Compare in-person diagnosis vs. remote diagnosis

At least four months after each patient`s initial examination, their information and lesion photographs were independently reviewed by two examiners using a web-based teleconsultation system. Both examiners analyzed the photos and information on their computers, as provided by the Department of Oral, Maxillofacial, and Dental Diseases for its professors. After reviewing the data, the examiners documented a final diagnosis for each patient. The level of agreement between the two examiners and between the examiners and the gold standard (in-person clinical oral examination) was assessed using Cohen's kappa coefficient. The interpretation of Cohen’s kappa coefficient is as follows: Perfect agreement (κ = 1), substantial agreement (0.81 ≤ κ < 1), moderate agreement (0.61 ≤ κ < 0.81), fair Agreement (0.41 ≤ κ < 0.61), slight agreement (0.21 ≤ κ < 0.41), and poor agreement (κ < 0.21). Data analysis was performed using IBM SPSS Statistics 22.

Result

Findings of phase 1: usability evaluation

Figure 2 illustrates the number of usability issues and their average severity ratings based on Nielsen’s severity rating scale.

The results revealed 66 usability problems. Most of these issues were associated with helping users recognize, diagnose, and recover from errors, as well as to help and documentation, with 11 being minor severity issues. The average severity scores for the identified heuristic violations ranged from 0 (indicating a good match between the system and the real world) to 3.00 (indicating issues with consistency, standards, aesthetics, and minimalist design). Scores closer to 0 suggest a more usable system (Fig. 3).

Fig. 3
figure 3

The number of usability problems and the average severity scores on Nielsen’s severity rating scale

Full size image

Findings of phase 2: reliability assessment

In this study, out of 109 participants, 63 (57.8%) were female and 46 (42.2%) were male, with an average age of 47 ± 19.46 years. Table 2 presents the concordance results between the gold standard and Examiner 1 and Examiner 2, as well as the internal agreement between the two examiners. The findings, detailed in Table 1, demonstrate that the web-based teleconsultation system performed substantially (0.81 ≤ κ < 1) with statistical significance (P > 0.05).

Table 2 Results of the concordance between face-to-face oral examination diagnosis and remote diagnosis
Full size table

Discussion

The usability evaluation revealed 66 issues, primarily concerning users` ability to recognize, diagnose, and recover from errors, as well as help and documentation. Nielsen's usability evaluation, employed in this study, assessed how well the user interface adhered to design principles, making it a useful tool for software and information system developers [43]. Similar heuristic evaluation methods have been applied in various studies to evaluate the usability of telehealth systems. For instance, Stein et al. (2016) utilized the heuristic method to evaluate a prototype mobile application for emergency triage in teledentistry, revealing usability issues related to stability and standards [44]. Lilleholt et al. (2015) employed Nielsen's exploratory evaluation method to evaluate a telehealth system, identifying key issues with system-world match, consistency and standards, and aesthetic minimalist design [45]. Khashe et al. (2021) used Nielsen's heuristic evaluation to assess a virtual visit system, noting usability issues related to error prevention, help and documentation, visibility of system status, system-world match, and user control [46]. While these studies differ in focus, they all consistently address usability issues.

The results highlight the significant performance of the web-based teleconsultation system. Key factors contributing to this performance include the use of the iPhone 13 Pro Max for capturing oral cavity photos. Improving photo quality is crucial for enhancing the accuracy and precision of remote diagnosis [14, 31]. This study also addressed essential requirements for oral cavity photography, such as proper patient positioning [25], adequate lighting [21, 25, 47], and using the back camera of a smartphone [25].

Examiners using this web-based teleconsultation system can access comprehensive and detailed patient information, including demographic data, clinical examinations, and laboratory test results. This access improves the accuracy of remote diagnoses and increases agreement between on-site and remote diagnoses. Having correct, accurate, timely and complete information is a key facilitator in the effectiveness of teledentistry [9].

The study`s findings aligned with those of Vinayagamoorthy et al. (2019), who also found significant agreement between diagnosing oral potentially malignant disorders via the WhatsApp application and face-to-face examination [31]. However, the level of agreement in our study was higher, which may be attributed to the use of different teledentistry platforms and variations in study methodologies. Our results are also aligned with Estai et al. (2017) who compared in-person and teledentistry methods for diagnosing dental caries using a Remote-I platform and found high agreement between the two methods [34]. Additionally, our findings are in line with those of AlShaya et al. (2020), supporting the acceptable reliability of telediagnosis of oral lesions through photographs and teledentistry platforms [35].

Despite these findings, there are some limitations to consider. The study protocol did not include measurement of sensitivity, specificity, positive predictive value, and negative predictive value. Consequently, reliability was assessed based on the calculation of Cohen's kappa statistic and the agreement between the two evaluators. Furthermore, the agreement between face-to-face and remote diagnosis was considered binary, with diagnoses categorized as either 0 or 1 based on an exact diagnostic match. The use of the iPhone 13 Pro Max likely improved the quality of oral lesion images, which may have positively impacted the reliability of the results.

Conclusion

In summary, the web-based teleconsultation system demonstrated substantial performance in remotely diagnosing oral lesions, making it a valuable alternative, especially during emergencies like the COVID-19 pandemic. When face-to-face visits are not possible for timely diagnosis or ongoing treatment, this system provides a practical solution. Moreover, it offers the convenience of remote follow-up sessions for patients residing in rural areas who might have difficulty attending in-person appointments.

The versatility of the system also facilitates consultations among oral medicine specialists, physicians, dentists, and other healthcare professionals, promoting collaborative decision-making and knowledge sharing.

However, it is crucial to recognize that the teleconsultation system faces challenges. The identification of various usability issues highlights the need to address these concerns to optimize the effectiveness and user experience of the system. Ongoing refinement and improvements in usability are essential for the system`s overall success and acceptance in the field of oral medicine.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to the need to protect the privacy of study participants, but are available from the corresponding author on reasonable request.

References

  1. Radwan-Oczko M, Sokół I, Babuśka K, Owczarek-Drabińska JE. Prevalence and characteristic of oral mucosa lesions. Symmetry. 2022;14(2):307.

    Article  Google Scholar 

  2. Ge S, Liu L, Zhou Q, Lou B, Zhou Z, Lou J, et al. Prevalence of and related risk factors in oral mucosa diseases among residents in the Baoshan District of Shanghai. China PeerJ. 2020;8:e8644.

    Article  PubMed  Google Scholar 

  3. Beaty CS, Short AG, Mewar P. Oral Mucosal Lesions, Immunologic Diseases. 2023 Nov 14. In: StatPearls [Internet]. Treasure Island: StatPearls Publishing; 2024.

  4. Chiang CP, Chang JYF, Wang YP, Wu YH, Lu SY, Sun A. Oral lichen planus–differential diagnoses, serum autoantibodies, hematinic deficiencies, and management. J Formos Med Assoc. 2018;117(9):756–65.

    Article  PubMed  Google Scholar 

  5. Glick M, Greenberg MS, Lockhart PB, Challacombe SJ. Introduction to oral medicine and oral diagnosis: patient evaluation. Burket's Oral Medicine. 13th ed. Wiley-Blackwell; 2021. p. 1–18.

  6. Zahid E, Bhatti O, Zahid MA, Stubbs M. Overview of common oral lesions. Malays Fam Physician. 2022;17(3):9.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Flores APdC, Lazaro SA, Molina-Bastos CG, Guattini VLdO, Umpierre RN, Gonçalves MR, et al. Teledentistry in the diagnosis of oral lesions: A systematic review of the literature. J Am Med Inform Assoc. 2020;27(7):1166–72.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Torres-Pereira CC, Morosini IdAC, Possebon RS, Giovanini AF, Bortoluzzi MC, Leao JC, et al. Teledentistry: distant diagnosis of oral disease using e-mails. Telemed Health. 2013;19(2):117–21.

    Article  Google Scholar 

  9. Niknam F, Sharifian R, Bashiri A, Mardani M, Akbari R, Tuffaha H, et al. Tele-dentistry, its trends, scope, and future framework in oral medicine; a scoping review during January 1999 to December 2021. Arch Pub Health. 2023;81(1):1–14.

    Google Scholar 

  10. Pérez González A, Gallas Torreira M, Chamorro Petronacci CM, Pérez Sayáns M. Teledentistry: a new approach in dental medicine. In Enhanced Telemedicine and e-Health: Advanced IoT Enabled Soft Computing Framework. Cham: Springer International Publishing; 2021. p. 41–64.

  11. Tesfalul M, Littman-Quinn R, Antwi C, Ndlovu S, Motsepe D, Phuthego M, et al. Evaluating the potential impact of a mobile telemedicine system on coordination of specialty care for patients with complicated oral lesions in Botswana. J Am Med Inform Assoc. 2016;23(e1):e142–5.

    Article  PubMed  Google Scholar 

  12. Goffin G, Carter N, Sari Widyarman A, Erri Astoeti T, Kabir Bulbul H, Puplampu P, et al. Role of teledentistry in enabling improved oral care outcomes. Br Dent J. 2024;236(3):162–8.

    Article  PubMed  Google Scholar 

  13. Di Fede O, Panzarella V, Buttacavoli F, La Mantia G, Campisi G. Doctoral: A smartphone-based decision support tool for the early detection of oral potentially malignant disorders. Digital Health. 2023;9:20552076231177140.

    PubMed  PubMed Central  Google Scholar 

  14. Birur NP, Gurushanth K, Patrick S, Sunny SP, Raghavan SA, Gurudath S, et al. Role of community health worker in a mobile health program for early detection of oral cancer. Indian J Cancer. 2019;56(2):107–13.

    Article  PubMed  Google Scholar 

  15. Kengne Talla P, Allison P, Bussières A, Giraudeau N, Komarova S, Basiren Q, et al. Teledentistry for improving access to, and quality of oral health care: A protocol for an overview of systematic reviews and meta-analyses. PLoS ONE. 2024;19(1):e0288677.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Alves FA, Saunders D, Sandhu S, Xu Y, de Mendonça NF, Treister NS. Implication of COVID-19 in oral oncology practices in Brazil, Canada, and the United States. Oral Dis. 2021;27(Suppl 3):793.

    Article  PubMed  Google Scholar 

  17. Bastani P, Niknam F, Rezazadeh M, Rossi-Fedele G, Edirippulige S, Samadbeik M. Dentistry website analysis: An overview of the content of formulated questions and answers. Heliyon. 2022;8:e10250.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Blomstrand L, Sand LP, Gullbrandsson L, Eklund B, Kildal M, Hirsch J-M. Telemedicine—A complement to traditional referrals in oral medicine. Telemedicine and e-Health. 2012;18(7):549–53.

    Article  PubMed  Google Scholar 

  19. Macken J, Fortune F, Buchanan J. Remote telephone clinics in oral medicine: reflections on the place of virtual clinics in a specialty that relies so heavily on visual assessment. A note of caution. Br J Oral and Maxillofac Surg. 2021;59(5):605–8.

    Article  Google Scholar 

  20. Fricton J, Chen H. Using teledentistry to improve access to dental care for the underserved. Dental Clinics. 2009;53(3):537–48.

    PubMed  Google Scholar 

  21. Haron N, Zain RB, Nabillah WM, Saleh A, Kallarakkal TG, Ramanathan A, et al. Mobile phone imaging in low resource settings for early detection of oral cancer and concordance with clinical oral examination. Telemedicine and e-Health. 2017;23(3):192–9.

    Article  PubMed  Google Scholar 

  22. Maret D, Warnakulasuriya S, Herbault-Barres B, Savall F, Vigarios E. Telemedicine contributing to an incidental finding of a premalignant lesion. Oral Oncol. 2021;118:105331.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Perdoncini NN, Schussel JL, Amenábar JM, Torres-Pereira CC. Use of smartphone video calls in the diagnosis of oral lesions: Teleconsultations between a specialist and patients assisted by a general dentist. J Am Dent Assoc. 2021;152(2):127–35.

    Article  PubMed  Google Scholar 

  24. Deshpande S, Patil D, Dhokar A, Bhanushali P, Katge F. Teledentistry: A boon amidst COVID-19 lockdown—a narrative review. Int J Telemed Appl. 2021;2021:1–6.

    Google Scholar 

  25. Maret D, Peters OA, Auria JP, Savall F, Vigarios E. Smartphone oral self-photography in teledentistry: recommendations for the patient. J Telemed Telecare. 2021;30:186–93.

    Article  PubMed  Google Scholar 

  26. Di Fede O, La Mantia G, Cimino MG, Campisi G. Protection of patient data in digital Oral and general health care: A scoping review with respect to the current regulations. Oral. 2023;3(2):155–65.

    Article  Google Scholar 

  27. Zhou MX, Johnson EF, Arce K, Gruwell SF. Teledentistry in the management of a non-Hodgkin’s lymphoma manifesting as a gingival swelling: A case report. Spec Care Dentist. 2022;42(1):86–90.

    Article  PubMed  Google Scholar 

  28. Lv N, Sun M, Polonowita A, Mei L, Guan G. Management of oral medicine emergencies during COVID-19: A study to develop practise guidelines. Journal of dental sciences. 2021;16(1):493–500.

    Article  PubMed  Google Scholar 

  29. Fonseca BB, Perdoncini NN, da Silva VC, Gueiros LAM, Carrard VC, Lemos CA Jr, et al. Telediagnosis of oral lesions using smartphone photography. Oral Dis. 2022;28(6):1573–9.

    Article  PubMed  Google Scholar 

  30. Gambino O, Lima F, Pirrone R, Ardizzone E, Campisi G, di Fede O. Second opinion system for intraoral lesions. In 2014 IEEE 27th International Symposium on Computer-Based Medical Systems. IEEE; 2014. p. 495–6.

  31. Vinayagamoorthy K, Acharya S, Kumar M, Pentapati KC, Acharya S. Efficacy of a remote screening model for oral potentially malignant disorders using a free messaging application: a diagnostic test for accuracy study. Aust J Rural Health. 2019;27(2):170–6.

    Article  PubMed  Google Scholar 

  32. Valizadeh-Haghi H, Valizadeh-Haghi S, Naslseraji N, Zandian H. Smartphone Photography as a Teledentistry Method to Evaluate Anterior Composite Restorations. Int J Dent. 2023;2023(1):3171140.

  33. Jeong J-S, Pang N-S, Choi Y, Park K-M, Kim T, Xu X, et al. Importance of photography education to improve image quality for accurate remote diagnoses in dental trauma patients: observational study. JMIR Mhealth Uhealth. 2020;8(3):e15152.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Estai M, Kanagasingam Y, Huang B, Shiikha J, Kruger E, Bunt S, et al. Comparison of a smartphone-based photographic method with face-to-face caries assessment: a mobile teledentistry model. Telemed e-Health. 2017;23(5):435–40.

    Article  Google Scholar 

  35. AlShaya MS, Assery MK, Pani SC. Reliability of mobile phone teledentistry in dental diagnosis and treatment planning in mixed dentition. J Telemed Telecare. 2020;26(1–2):45–52.

    Article  PubMed  Google Scholar 

  36. Jeddi FR, Nabovati E, Bigham R, Farrahi R. Usability evaluation of a comprehensive national health information system: A heuristic evaluation. Informatics in Medicine Unlocked. 2020;19:100332.

    Article  Google Scholar 

  37. Martins, AI, Queirós A, Silva AG, Rocha NP. Usability Evaluation Methods: A Systematic Review. Human-Computer Interaction: Concepts, Methodologies, Tools, and Applications. 2016. p. 613–36.

  38. Koohjani Z, Aslani A, Abasi S, Kyiani S. A comprehensive tool for usability evaluation of telehealth. In pHealth 2019. IOS Press; 2019. p. 168–73.

  39. Kasim HF, Salih AI, Attash FM. Usability of telehealth among healthcare providers during COVID-19 pandemic in Nineveh Governorate, Iraq. Public Health Pract. 2023;5:100368.

  40. Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Informatics. 2009;78(5):340–53.

    Article  Google Scholar 

  41. Donner A, Eliasziw M. A goodness-of-fit approach to inference procedures for the kappa statistic: confidence and sample size estimation interval construction, significance-testing. Stat Med. 1992;11(11):1511–9.

    Article  PubMed  Google Scholar 

  42. Shoukri MM, Asyali M, Donner A. Sample size requirements for the design of reliability study: review and new results. Stat Methods Med Res. 2004;13(4):251–71.

    Article  Google Scholar 

  43. Nielsen J. Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. New York: Association for Computing Machinery; 1994. p. 152–158.

  44. Stein CD, Xiao X, Levine S, Schleyer TK, Hochheiser H, Thyvalikakath TP. A prototype mobile application for triaging dental emergencies. J Am Dent Assoc. 2016;147(10):782–91. e1.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Lilholt PH, Jensen MH, Hejlesen OK. Heuristic evaluation of a telehealth system from the Danish TeleCare North Trial. Int J Med Informatics. 2015;84(5):319–26.

    Article  Google Scholar 

  46. Khashe Y, Tabibzadeh M, Zhou D. Improving Quality of Care in Virtual Visits: Heuristic Evaluations of the User Interface in Telehealth. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 65. Los Angeles: SAGE Publications; 2021. p. 1347–51.

  47. Carrard V, Roxo Gonçalves M, Rodriguez Strey J, Pilz C, Martins M, Martins M, et al. Telediagnosis of oral lesions in primary care: The EstomatoNet Program. Oral Dis. 2018;24(6):1012–9.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors wish to thank all study participants for their valuable cooperation. Additionally, the authors would like to thank the professors, residents, and staff of Oral, Maxillofacial, and Dental Diseases Department at the School of Dentistry, Shiraz University of Medical Science, for their assistance and support.

Funding

No funding.

Author information

Authors and Affiliations

  1. Department of Health Information Management, Iran University of Medical Science, Tehran,, Iran

    Fatemeh Niknam, Azadeh Bashiri & Roxana Sharifian

  2. Department of Health Information Management, Shiraz University of Medical Sciences, Shiraz, Iran

    Maryam Mardani

  3. College of Business, Government and Law, Flinders University, Adelaide, Bedford Park, SA, 5042, Australia

    Peivand Bastani

  4. School of Dentistry, UQ Oral Health Centre, The University of Queensland, Brisbane, QLD, 4006, Australia

    Diep Ha

  5. Student Research Committee, School of Dentistry, Shiraz University of Medical Sciences, Shiraz, Iran

    Asma Sookhakian

  6. Department of Computer Engineering and Information Technology, Shiraz University of Technology, Shiraz, Iran

    Reza Akbari

Authors
  1. Fatemeh Niknam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maryam Mardani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peivand Bastani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Azadeh Bashiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Diep Ha
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Asma Sookhakian
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Reza Akbari
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Roxana Sharifian
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

FN performed data collection, data analysis, writing the manuscript, contributed in the study design, and trained and supervised residents and professors to use teleconsultation system, MM was clinical advisor and contributed in clinical examination and study design, R SH and AB were supervisors and contributed in the study design, PB performed English language editing and critical revision of the manuscript, DH performed critical revision of the manuscript, AS contributed in data collection, RA was a technical advisor and confirmed the technical specifications of the teleconsultation system.

Corresponding author

Correspondence to Roxana Sharifian.

Ethics declarations

Ethics approval and consent to participate

The study protocol was approved by the Research Ethics Committee of Shiraz University of Medical Science, Iran, under code IR.SUMS.DENTAL.REC.1401.072. All methods were carried out according to the relevant guidelines and regulations. Informed consent was obtained from all the participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Niknam, F., Mardani, M., Bastani, P. et al. Assessing the usability and reliability of a web-based teledentistry tool for remote diagnosis of oral lesions: a cross-sectional study. BMC Oral Health 24, 1094 (2024). https://doi.org/10.1186/s12903-024-04696-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12903-024-04696-7

Keywords

BMC Oral Health

ISSN: 1472-6831

Contact us