Evaluation of an intelligent virtual assistant (chatbot) in cardiopulmonary resuscitation
DOI:
https://doi.org/10.59681/2175-4411.v16.iEspecial.2024.1256Keywords:
Health Education, Artificial Intelligence, Cardiopulmonary ResuscitationAbstract
Objective: Evaluate a chatbot to assist in learning the Cardiopulmonary Resuscitation Procedure (CPR). Methods: Cross-sectional, prospective, analytical and observational study. Validation occurred through evaluation by experts, using the Content Validation Index (CVI) and the Delphi method. Validation by the end user occurred through usability evaluations, which used the Single Ease Question instrument, the System Usability Scale protocol and user experience, in which the User Experience Questionnaire was used. Results: The experts validated the chatbot's content using scores assigned to some questions, with an average CVI of 0.942. The calculated overall relative efficiency was 90.80% and satisfaction achieved excellent results, with a SUS score of 80.875. Conclusions: There was a high degree of agreement among professionals, a good increase in users' knowledge about CPR and few difficulties in using the chatbot, despite limiting factors.
References
Hoermann S, McCabe KL, Milne DN, Calvo RA. Application of Synchronous Text-Based Dialogue Systems in Mental Health Interventions: Systematic Review. J Med Internet Res [Internet]. 1o de agosto de 2017 [citado 29 de janeiro de 2024];19(8). Disponível em: https://pubmed.ncbi.nlm.nih.gov/28784594/
Rocha JSY. Uso de tecnologias da informação e comunicação na educação em saúde. Medicina (Ribeirão Preto). 2015;48(3):214–23.
De Campos Filho AS, Vasconcelos Cursino JR, Do Nascimento JWA, De Souza RR, Da Silva Lima Roque G, De Souza Cavalcanti AR. Content and Usability Validation of an Intelligent Virtual Conversation Assistant Used for Virtual Triage During the COVID-19 Pandemic in Brazil. Comput Inform Nurs [Internet]. 2 de novembro de 2022 [citado 29 de janeiro de 2024];40(11):779–85. Disponível em: https://pubmed.ncbi.nlm.nih.gov/35234699/
Holmes S, Moorhead A, Bond R, Zheng H, Coates V, Mctear M. Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces? Em: Proceedings of the 31st European Conference on Cognitive Ergonomics [Internet]. New York, NY, USA: Association for Computing Machinery; 2019. p. 207–14. (ECCE ’19). Disponível em: https://doi.org/10.1145/3335082.3335094
Clarck DE. R A Cowley, the “Golden Hour,” the “Momentary Pause,” and the “Third Space” - PubMed. Am Surg [Internet]. 1o de dezembro de 2017 [citado 29 de janeiro de 2024];83(12):1401–6. Disponível em: https://pubmed.ncbi.nlm.nih.gov/29336762/
Pasquali L. Psicometria: Teoria dos testes na psicologia e na educação. São Paulo; 2013.
ISO 9241-11:2018(en), Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts [Internet]. [citado 28 de agosto de 2022]. Disponível em: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en
Laugwitz B, Held T, Schrepp M. Construction and evaluation of a user experience questionnaire. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) [Internet]. 2008 [citado 29 de janeiro de 2024];5298 LNCS:63–76. Disponível em: https://link.springer.com/chapter/10.1007/978-3-540-89350-9_6
Coluci MZO, Alexandre NMC, Milani D. Construção de instrumentos de medida na área da saúde. Cien Saude Colet [Internet]. março de 2015 [citado 29 de janeiro de 2024];20(3):925–36. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232015000300925&lng=pt&tlng=pt
Lynn MR. Determination and quantification of content validity. 1986 [citado 29 de janeiro de 2024];35(6):382–5. Disponível em: https://pubmed.ncbi.nlm.nih.gov/3640358/
Grant JS, Davis LL. Selection and use of content experts for instrument development. Res Nurs Health [Internet]. junho de 1997 [citado 29 de janeiro de 2024];20(3):269–74. Disponível em: https://onlinelibrary.wiley.com/doi/10.1002/(SICI)1098-240X(199706)20:3<269::AID-NUR9>3.0.CO;2-G
Merchant RM, Topjian AA, Panchal AR, Cheng A, Aziz K, Berg KM, et al. Part 1: Executive Summary: 2020 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation [Internet]. 20 de outubro de 2020 [citado 29 de janeiro de 2024];142(2):S337–57. Disponível em: https://www.ahajournals.org/doi/abs/10.1161/CIR.0000000000000918
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Submission of a paper to Journal of Health Informatics is understood to imply that it is not being considered for publication elsewhere and that the author(s) permission to publish his/her (their) article(s) in this Journal implies the exclusive authorization of the publishers to deal with all issues concerning the copyright therein. Upon the submission of an article, authors will be asked to sign a Copyright Notice. Acceptance of the agreement will ensure the widest possible dissemination of information. An e-mail will be sent to the corresponding author confirming receipt of the manuscript and acceptance of the agreement.
